Reproducible documentation of analyses
Does Computer-Based Feedback Enhance Revision Performance in Writing Non-Fictional Texts? A Meta-Analysis
We folded the code for better readability.
You can display the code of our analyses
- by clicking on “Code” > “Show All Code” in the top right corner or
- by clicking in “Code” in each section of the document individually
Import Data Sets
Two data sets were used:
- Data set “dataN33_prepost”: studies with pre-post design
- Data set “dataN14_control”: studies with pre-post-control design
Download the data (embeded in this HTML file):
Pre-Post comparison
Code
# prepare data set for meta-analysis
data_pp <- data.frame(
sampleNr = 1:33,
id = c(1,1,1,2,2,3,3,4,4,4,4,4,4,5,5,6,7,7,8,8,9,10,11,12,12,13,14,14,14,15,15, 16, 17),
m_pre = dataN33_prepost$M_pre,
m_post = dataN33_prepost$M_post,
sd_pre = dataN33_prepost$SD_pre,
sd_post = dataN33_prepost$SD_post,
ni = dataN33_prepost$N,
slab = dataN33_prepost$author)Meta-analytic effect
As we did not find the pre-post test correlation reported in any of the studies, we ran a sensitivity analysis.
One of the studies with control group design provided open data (Burkhart et al., 2020) so that we were able to calculate its pre-post-test correlations (r = .72), which we used as an anchor for the sensitivity analysis.
We included a broad range of 26 (conservative) plausible values in the sensitivity analyses with correlations from r= .50 - .75, using steps of .01.
This will produce results from 26 meta-analyses. By the variation of results, we are able to estimate the robustness of our analyses.
Code
# Establish empty data frame to be filled with results
sensitivity <- data.frame(ri_t = as.numeric(), # assumed pre-post correlation
beta = as.numeric(), # meta-analytic ES
pvalue = as.numeric(), # p value of ES
se = as.numeric(), # SE of meta-analytic ES
sigma2 = as.numeric(), # Tau squared
yi.f01 = as.numeric(), # ES of individual studies
yi.f02 = as.numeric(),
yi.f03 = as.numeric(),
yi.f04 = as.numeric(),
yi.f05 = as.numeric(),
yi.f06 = as.numeric(),
yi.f07 = as.numeric(),
yi.f08 = as.numeric(),
yi.f09 = as.numeric(),
yi.f10 = as.numeric(),
yi.f11 = as.numeric(),
yi.f12 = as.numeric(),
yi.f13 = as.numeric(),
yi.f14 = as.numeric(),
yi.f15 = as.numeric(),
yi.f16 = as.numeric(),
yi.f17 = as.numeric(),
yi.f18 = as.numeric(),
yi.f19 = as.numeric(),
yi.f20 = as.numeric(),
yi.f21 = as.numeric(),
yi.f22 = as.numeric(),
yi.f23 = as.numeric(),
yi.f24 = as.numeric(),
yi.f25 = as.numeric(),
yi.f26 = as.numeric(),
yi.f27 = as.numeric(),
yi.f28 = as.numeric(),
yi.f29 = as.numeric(),
yi.f30 = as.numeric(),
yi.f31 = as.numeric(),
yi.f32 = as.numeric(),
yi.f33 = as.numeric(),
sei.f01 = as.numeric(), # SEs of ES of individual studies
sei.f02 = as.numeric(),
sei.f03 = as.numeric(),
sei.f04 = as.numeric(),
sei.f05 = as.numeric(),
sei.f06 = as.numeric(),
sei.f07 = as.numeric(),
sei.f08 = as.numeric(),
sei.f09 = as.numeric(),
sei.f10 = as.numeric(),
sei.f11 = as.numeric(),
sei.f12 = as.numeric(),
sei.f13 = as.numeric(),
sei.f14 = as.numeric(),
sei.f15 = as.numeric(),
sei.f16 = as.numeric(),
sei.f17 = as.numeric(),
sei.f18 = as.numeric(),
sei.f19 = as.numeric(),
sei.f20 = as.numeric(),
sei.f21 = as.numeric(),
sei.f22 = as.numeric(),
sei.f23 = as.numeric(),
sei.f24 = as.numeric(),
sei.f25 = as.numeric(),
sei.f26 = as.numeric(),
sei.f27 = as.numeric(),
sei.f28 = as.numeric(),
sei.f29 = as.numeric(),
sei.f30 = as.numeric(),
sei.f31 = as.numeric(),
sei.f32 = as.numeric(),
sei.f33 = as.numeric()
)
# starting loop over 26 possible pre-post-correlations
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
# compute the meta-analysis
rma_overall_clustered <- rma.mv(yi, vi,
data=data_pp_es,
random = ~ 1 | id # take clustered data into account
)
# save estimates for sensitivity analysis
sensitivity <- sensitivity %>%
add_row(ri_t = ri_t,
pvalue = rma_overall_clustered$pval,
beta = rma_overall_clustered$beta[,1],
se = rma_overall_clustered$se[1],
sigma2 = rma_overall_clustered$sigma2[1],
yi.f01 = rma_overall_clustered$yi.f[1],
yi.f02 = rma_overall_clustered$yi.f[2],
yi.f03 = rma_overall_clustered$yi.f[3],
yi.f04 = rma_overall_clustered$yi.f[4],
yi.f05 = rma_overall_clustered$yi.f[5],
yi.f06 = rma_overall_clustered$yi.f[6],
yi.f07 = rma_overall_clustered$yi.f[7],
yi.f08 = rma_overall_clustered$yi.f[8],
yi.f09 = rma_overall_clustered$yi.f[9],
yi.f10 = rma_overall_clustered$yi.f[10],
yi.f11 = rma_overall_clustered$yi.f[11],
yi.f12 = rma_overall_clustered$yi.f[12],
yi.f13 = rma_overall_clustered$yi.f[13],
yi.f14 = rma_overall_clustered$yi.f[14],
yi.f15 = rma_overall_clustered$yi.f[15],
yi.f16 = rma_overall_clustered$yi.f[16],
yi.f17 = rma_overall_clustered$yi.f[17],
yi.f18 = rma_overall_clustered$yi.f[18],
yi.f19 = rma_overall_clustered$yi.f[19],
yi.f20 = rma_overall_clustered$yi.f[20],
yi.f21 = rma_overall_clustered$yi.f[21],
yi.f22 = rma_overall_clustered$yi.f[22],
yi.f23 = rma_overall_clustered$yi.f[23],
yi.f24 = rma_overall_clustered$yi.f[24],
yi.f25 = rma_overall_clustered$yi.f[25],
yi.f26 = rma_overall_clustered$yi.f[26],
yi.f27 = rma_overall_clustered$yi.f[27],
yi.f28 = rma_overall_clustered$yi.f[28],
yi.f29 = rma_overall_clustered$yi.f[29],
yi.f30 = rma_overall_clustered$yi.f[30],
yi.f31 = rma_overall_clustered$yi.f[31],
yi.f32 = rma_overall_clustered$yi.f[32],
yi.f33 = rma_overall_clustered$yi.f[33],
sei.f01 = sqrt(data_pp_es$vi[1]),
sei.f02 = sqrt(data_pp_es$vi[2]),
sei.f03 = sqrt(data_pp_es$vi[3]),
sei.f04 = sqrt(data_pp_es$vi[4]),
sei.f05 = sqrt(data_pp_es$vi[5]),
sei.f06 = sqrt(data_pp_es$vi[6]),
sei.f07 = sqrt(data_pp_es$vi[7]),
sei.f08 = sqrt(data_pp_es$vi[8]),
sei.f09 = sqrt(data_pp_es$vi[9]),
sei.f10 = sqrt(data_pp_es$vi[10]),
sei.f11 = sqrt(data_pp_es$vi[11]),
sei.f12 = sqrt(data_pp_es$vi[12]),
sei.f13 = sqrt(data_pp_es$vi[13]),
sei.f14 = sqrt(data_pp_es$vi[14]),
sei.f15 = sqrt(data_pp_es$vi[15]),
sei.f16 = sqrt(data_pp_es$vi[16]),
sei.f17 = sqrt(data_pp_es$vi[17]),
sei.f18 = sqrt(data_pp_es$vi[18]),
sei.f19 = sqrt(data_pp_es$vi[19]),
sei.f20 = sqrt(data_pp_es$vi[20]),
sei.f21 = sqrt(data_pp_es$vi[21]),
sei.f22 = sqrt(data_pp_es$vi[22]),
sei.f23 = sqrt(data_pp_es$vi[23]),
sei.f24 = sqrt(data_pp_es$vi[24]),
sei.f25 = sqrt(data_pp_es$vi[25]),
sei.f26 = sqrt(data_pp_es$vi[26]),
sei.f27 = sqrt(data_pp_es$vi[27]),
sei.f28 = sqrt(data_pp_es$vi[28]),
sei.f29 = sqrt(data_pp_es$vi[29]),
sei.f30 = sqrt(data_pp_es$vi[30]),
sei.f31 = sqrt(data_pp_es$vi[31]),
sei.f32 = sqrt(data_pp_es$vi[32]),
sei.f33 = sqrt(data_pp_es$vi[33])
)
}Overview of meta-analytic ES
Results from all 26 meta-analyses:
- ri_t: assumed pre-post-correlation
- beta: meta-analytic ES
- se: SE of meta-analytic ES
- pvalue: p value of meta-analytic ES
Code
| ri_t | beta | se | pvalue | sigma2 |
|---|---|---|---|---|
| 0.50 | 0.4609693 | 0.1568736 | 0.0032983 | 0.3977544 |
| 0.51 | 0.4609604 | 0.1568797 | 0.0033001 | 0.3980426 |
| 0.52 | 0.4609474 | 0.1568848 | 0.0033020 | 0.3983259 |
| 0.53 | 0.4609302 | 0.1568888 | 0.0033040 | 0.3986040 |
| 0.54 | 0.4609085 | 0.1568918 | 0.0033061 | 0.3988767 |
| 0.55 | 0.4608822 | 0.1568937 | 0.0033082 | 0.3991438 |
| 0.56 | 0.4608511 | 0.1568944 | 0.0033105 | 0.3994050 |
| 0.57 | 0.4608150 | 0.1568940 | 0.0033129 | 0.3996600 |
| 0.58 | 0.4607737 | 0.1568922 | 0.0033153 | 0.3999087 |
| 0.59 | 0.4607270 | 0.1568891 | 0.0033179 | 0.4001507 |
| 0.60 | 0.4606747 | 0.1568847 | 0.0033206 | 0.4003856 |
| 0.61 | 0.4606165 | 0.1568788 | 0.0033234 | 0.4006133 |
| 0.62 | 0.4605521 | 0.1568714 | 0.0033263 | 0.4008332 |
| 0.63 | 0.4604813 | 0.1568625 | 0.0033293 | 0.4010451 |
| 0.64 | 0.4604038 | 0.1568518 | 0.0033325 | 0.4012485 |
| 0.65 | 0.4603192 | 0.1568395 | 0.0033358 | 0.4014431 |
| 0.66 | 0.4602273 | 0.1568253 | 0.0033392 | 0.4016284 |
| 0.67 | 0.4601277 | 0.1568092 | 0.0033428 | 0.4018038 |
| 0.68 | 0.4600200 | 0.1567911 | 0.0033466 | 0.4019689 |
| 0.69 | 0.4599038 | 0.1567709 | 0.0033505 | 0.4021232 |
| 0.70 | 0.4597787 | 0.1567485 | 0.0033546 | 0.4022660 |
| 0.71 | 0.4596442 | 0.1567237 | 0.0033589 | 0.4023966 |
| 0.72 | 0.4594999 | 0.1566964 | 0.0033633 | 0.4025145 |
| 0.73 | 0.4593452 | 0.1566664 | 0.0033679 | 0.4026188 |
| 0.74 | 0.4591795 | 0.1566337 | 0.0033728 | 0.4027088 |
| 0.75 | 0.4590023 | 0.1565980 | 0.0033778 | 0.4027835 |
We summarize these results:
- ES_mean: meta-analytic ES (mean over the 26 meta-analyses)
- ES_min: meta-analytic ES (min of the 26 meta-analyses)
- ES_max: meta-analytic ES (max of the 26 meta-analyses)
- CI_lower_mean: lower CI of meta-analytic ES (mean over the 26 meta-analyses)
- CI_lower_min: lower CI of meta-analytic ES (min of the 26 meta-analyses)
- CI_lower_max: lower CI of meta-analytic ES (max of the 26 meta-analyses)
- CI_upper_mean: upper CI of meta-analytic ES (mean over the 26 meta-analyses)
- CI_upper_min: upper CI of meta-analytic ES (min of the 26 meta-analyses)
- CI_upper_max: upper CI of meta-analytic ES (max of the 26 meta-analyses)
- pvalue_mean: p value of meta-analytic ES (mean over the 26 meta-analyses)
- pvalue_min: p value of meta-analytic ES (min of the 26 meta-analyses)
- pvalue_max: p value of meta-analytic ES (max of the 26 meta-analyses)
Code
# compute mean, min and max of ES, CI and pvalue from meta-analysis
sensitivity %>%
dplyr::summarise(ES_mean = mean(beta),
ES_min = min(beta),
ES_max = max(beta),
CI_lower_mean = mean(beta-(1.96*se)),
CI_lower_min = min(beta-(1.96*se)),
CI_lower_max = max(beta-(1.96*se)),
CI_upper_mean = mean(beta+(1.96*se)),
CI_upper_min = min(beta+(1.96*se)),
CI_upper_max = max(beta+(1.96*se)),
pvalue_mean = mean(pvalue),
pvalue_min = min(pvalue),
pvalue_max = max(pvalue),
sigma2_mean = mean(sigma2)) %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover")) %>%
column_spec(1, background = "#C74A48") %>%
column_spec(2, background = "#B54341") %>%
column_spec(3, background = "#913634") %>%
column_spec(4, background = "#27A357") %>%
column_spec(5, background = "#208748") %>%
column_spec(6, background = "#186636") %>%
column_spec(7, background = "#B244B8") %>%
column_spec(8, background = "#8A358F") %>%
column_spec(9, background = "#6A296E") %>%
column_spec(10, background = "#9E8A47") %>%
column_spec(11, background = "#8F7C40") %>%
column_spec(12, background = "#6E6031")| ES_mean | ES_min | ES_max | CI_lower_mean | CI_lower_min | CI_lower_max | CI_upper_mean | CI_upper_min | CI_upper_max | pvalue_mean | pvalue_min | pvalue_max | sigma2_mean |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.4603285 | 0.4590023 | 0.4609693 | 0.1529602 | 0.1520702 | 0.1534971 | 0.7676968 | 0.7659345 | 0.7684446 | 0.0033314 | 0.0032983 | 0.0033778 | 0.400702 |
We summarize the same aspects for all 33 studies:
Note that the ES does not vary due to different pre-post-correlation, but SE does.
Code
# compute mean, min and max of ES and CI from each study
sensitivity %>%
dplyr::select(-c(beta, pvalue, se, sigma2)) %>%
pivot_longer(c(2:67), # reshape data from
names_to = "variable", # sensitivity analysis
values_to = "values") %>%
mutate(sampleNr = as.numeric(str_sub(variable, -2, -1)),
variable = str_sub(variable, 1, -5)) %>%
pivot_wider(id_cols = c(sampleNr, ri_t),
names_from = "variable",
values_from = "values") %>%
group_by(sampleNr) %>%
dplyr::summarise(ES = mean(yi),
CI_lower_mean = mean(yi-(1.96*sei)),
CI_lower_min = min(yi-(1.96*sei)),
CI_lower_max = max(yi-(1.96*sei)),
CI_upper_mean = mean(yi+(1.96*sei)),
CI_upper_min = min(yi+(1.96*sei)),
CI_upper_max = max(yi+(1.96*sei))) %>%
right_join(data_pp[c("sampleNr", "slab")],., by = "sampleNr") %>%
dplyr::select(-sampleNr) %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover")) %>%
column_spec(2, background = "#C74A48") %>%
column_spec(3, background = "#27A357") %>%
column_spec(4, background = "#208748") %>%
column_spec(5, background = "#186636") %>%
column_spec(6, background = "#B244B8") %>%
column_spec(7, background = "#8A358F") %>%
column_spec(8, background = "#6A296E")| slab | ES | CI_lower_mean | CI_lower_min | CI_lower_max | CI_upper_mean | CI_upper_min | CI_upper_max |
|---|---|---|---|---|---|---|---|
| Burkhart et al., 2020 | 1.4692438 | 0.9394991 | 0.9039832 | 0.9766106 | 1.9989885 | 1.9618770 | 2.0345044 |
| Burkhart et al., 2020 | 0.7175647 | 0.3085082 | 0.2592778 | 0.3618768 | 1.1266212 | 1.0732525 | 1.1758516 |
| Burkhart et al., 2020 | 0.6060285 | 0.2358310 | 0.1878340 | 0.2882082 | 0.9762260 | 0.9238488 | 1.0242230 |
| Frost, 2008 | -0.5270188 | -1.0024157 | -1.0670867 | -0.9315185 | -0.0516220 | -0.1225191 | 0.0130491 |
| Frost, 2008 | 0.3241156 | -0.0403714 | -0.0951686 | 0.0203292 | 0.6886025 | 0.6279020 | 0.7433997 |
| Kellogg et al., 2010 | 2.6877490 | 1.7725416 | 1.7465410 | 1.7990238 | 3.6029564 | 3.5764742 | 3.6289570 |
| Kellogg et al., 2010 | 2.9482961 | 1.9590182 | 1.9349339 | 1.9834837 | 3.9375740 | 3.9131084 | 3.9616582 |
| Lachner et al., 2017a | 0.1305370 | -0.0428142 | -0.0703551 | -0.0121139 | 0.3038881 | 0.2731878 | 0.3314290 |
| Lachner et al., 2017a | 0.1320125 | -0.0615734 | -0.0923213 | -0.0272991 | 0.3255983 | 0.2913241 | 0.3563462 |
| Lachner et al., 2017a | 0.1904274 | -0.0031278 | -0.0335017 | 0.0306819 | 0.3839826 | 0.3501730 | 0.4143566 |
| Lachner et al., 2017a | 0.0688945 | -0.1037383 | -0.1313844 | -0.0728916 | 0.2415273 | 0.2106806 | 0.2691734 |
| Lachner et al., 2017a | 0.0900085 | -0.1029692 | -0.1338062 | -0.0685713 | 0.2829863 | 0.2485883 | 0.3138232 |
| Lachner et al., 2017a | 0.1584356 | -0.0344010 | -0.0648790 | -0.0004473 | 0.3512723 | 0.3173186 | 0.3817503 |
| Lachner et al., 2017b (Study 2) | 1.4283419 | 0.8598146 | 0.8204439 | 0.9010153 | 1.9968692 | 1.9556685 | 2.0362399 |
| Lachner et al., 2017b (Study 3) | 0.7019564 | 0.1608947 | 0.0950690 | 0.2323198 | 1.2430181 | 1.1715929 | 1.3088438 |
| Lachner & Neuburg, 2019 | 1.0366435 | 0.6379179 | 0.6001713 | 0.6781094 | 1.4353692 | 1.3951777 | 1.4731158 |
| McCarthy et al., 2019 | 0.2243693 | 0.0026534 | -0.0318304 | 0.0409983 | 0.4460852 | 0.4077403 | 0.4805690 |
| McCarthy et al., 2019 | 0.2002616 | -0.0225713 | -0.0574544 | 0.0162464 | 0.4230945 | 0.3842768 | 0.4579775 |
| Palermo, 2017 | 0.4423406 | 0.3333490 | 0.3178205 | 0.3504521 | 0.5513322 | 0.5342292 | 0.5668608 |
| Palermo, 2017 | 0.9712657 | 0.8436314 | 0.8309127 | 0.8572206 | 1.0988999 | 1.0853107 | 1.1116186 |
| Roscoe et al., 2013 | 0.1394781 | -0.1564306 | -0.2033695 | -0.1041167 | 0.4353869 | 0.3830729 | 0.4823257 |
| Wang et al., 2020 | 0.1457761 | 0.0035404 | -0.0189958 | 0.0286539 | 0.2880118 | 0.2628983 | 0.3105480 |
| Zhu et al., 2017 | 0.3516930 | 0.2036157 | 0.1816037 | 0.2279688 | 0.4997703 | 0.4754171 | 0.5217822 |
| Wilson et al., 2023 | 0.7187023 | 0.4261396 | 0.3909572 | 0.4642767 | 1.0112650 | 0.9731279 | 1.0464473 |
| Wilson et al., 2023 | 0.7724223 | 0.4740418 | 0.4395019 | 0.5113638 | 1.0708027 | 1.0334807 | 1.1053426 |
| Correnti et al., 2022 | 0.4140777 | 0.3046662 | 0.2888570 | 0.3221039 | 0.5234892 | 0.5060515 | 0.5392984 |
| Wilson et al., 2022 | 0.2918232 | 0.1597651 | 0.1396677 | 0.1820579 | 0.4238813 | 0.4015885 | 0.4439788 |
| Wilson et al., 2022 | -0.0599990 | -0.1833126 | -0.2030753 | -0.1612600 | 0.0633146 | 0.0412621 | 0.0830773 |
| Wilson et al., 2022 | 0.0384150 | -0.0784551 | -0.0972110 | -0.0575224 | 0.1552851 | 0.1343524 | 0.1740410 |
| McCarthy et al., 2022 | 0.1974450 | -0.0234412 | -0.0580441 | 0.0150680 | 0.4183312 | 0.3798220 | 0.4529341 |
| McCarthy et al., 2022 | 0.1570233 | -0.0646667 | -0.0997150 | -0.0256202 | 0.3787133 | 0.3396668 | 0.4137616 |
| Butterfuss et al., 2022 | 0.2954736 | 0.0520840 | 0.0150924 | 0.0931104 | 0.5388631 | 0.4978368 | 0.5758548 |
| Niloy et al., 2023 | -0.6649928 | -0.7761696 | -0.7900395 | -0.7610866 | -0.5538160 | -0.5688990 | -0.5399460 |
Forest Plot
In order to be able to display a forest plot, we calculated a meta-analysis with the mean assumed pre-post-correlation (r=.625).
Code
## FOREST PLOT with mean correlation
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(.625, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
REM <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id)
###FOREST PLOT
forest(REM)Bias estimation
Funnel plot
Code
##Funnel plot
funnel(REM, legend = T)Trim and fill
Code
Estimated number of missing studies on the left side: 0 (SE = 3.1132)
Random-Effects Model (k = 33; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0.3068 (SE = 0.0833)
tau (square root of estimated tau^2 value): 0.5539
I^2 (total heterogeneity / total variability): 97.05%
H^2 (total variability / sampling variability): 33.86
Test for Heterogeneity:
Q(df = 32) = 602.8700, p-val < .0001
Model Results:
estimate se zval pval ci.lb ci.ub
0.4364 0.1008 4.3296 <.0001 0.2389 0.6340 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Code
funnel(taf_overall, legend = T)Egger’s test
Code
##Egger's test
regtest(rma_trimfill)
Regression Test for Funnel Plot Asymmetry
Model: mixed-effects meta-regression model
Predictor: standard error
Test for Funnel Plot Asymmetry: z = 6.2669, p < .0001
Limit Estimate (as sei -> 0): b = -0.2622 (CI: -0.5056, -0.0188)
Heterogeneity
Calculating \(I^2\)
Code
###HETEROGENEITY
# Establish empty data frame to be filled with results
heterogeneity_sen <- data.frame(ri_t = as.numeric(), # assumed pre-post correlation
I2 = as.numeric()) # I²
# starting loop over 26 possible pre-post-correlations
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
# compute the meta-analysis
REM <- rma.mv(yi, vi,
data=data_pp_es,
random = ~ 1 | id # take clustered data into account
)
# Formula
W <- diag(1/REM$vi)
X <- model.matrix(REM)
P <- W-W%*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W
I2 <- 100*REM$sigma2/(sum(REM$sigma2)+(REM$k - REM$p)/sum(diag(P)))
# save estimates for sensitivity analysis
heterogeneity_sen <- heterogeneity_sen %>%
add_row(ri_t = ri_t,
I2 = I2)
}
heterogeneity_sen %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | I2 |
|---|---|
| 0.50 | 97.03884 |
| 0.51 | 97.09374 |
| 0.52 | 97.14861 |
| 0.53 | 97.20346 |
| 0.54 | 97.25829 |
| 0.55 | 97.31310 |
| 0.56 | 97.36789 |
| 0.57 | 97.42267 |
| 0.58 | 97.47743 |
| 0.59 | 97.53219 |
| 0.60 | 97.58694 |
| 0.61 | 97.64169 |
| 0.62 | 97.69644 |
| 0.63 | 97.75119 |
| 0.64 | 97.80595 |
| 0.65 | 97.86072 |
| 0.66 | 97.91550 |
| 0.67 | 97.97031 |
| 0.68 | 98.02514 |
| 0.69 | 98.08000 |
| 0.70 | 98.13489 |
| 0.71 | 98.18983 |
| 0.72 | 98.24482 |
| 0.73 | 98.29987 |
| 0.74 | 98.35499 |
| 0.75 | 98.41018 |
Code
skim(heterogeneity_sen)| Name | heterogeneity_sen |
| Number of rows | 26 |
| Number of columns | 2 |
| _______________________ | |
| Column type frequency: | |
| numeric | 2 |
| ________________________ | |
| Group variables | None |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| ri_t | 0 | 1 | 0.625 | 0.076 | 0.500 | 0.562 | 0.625 | 0.688 | 0.75 | ▇▇▇▇▇ |
| I2 | 0 | 1 | 97.724 | 0.419 | 97.039 | 97.382 | 97.724 | 98.066 | 98.41 | ▇▇▇▇▇ |
Moderators
We will report results from all 26 meta-analyses for the moderators.
For each moderator we provide
- a table with the main parameters from each meta-analysis
- ri_t: assumed pre-post-correlation (sensitivity analysis)
- beta: ES of moderator
- ci.lb: lower bound CI of ES of moderator
- ci.ub: upper bound CI of ES of moderator
- pvalue: p value of ES of moderator
- a summary table of these parameters
Representation
Graphical representation
Code
#graphical representation
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$rep_g <- as.factor(dataN33_prepost$rep_graphical)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = rep_g)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.0475445 | 0.0900853 | -0.1290194 | 0.2241085 | 0.5976573 | 0.3899507 |
| 0.51 | 0.0467906 | 0.0892401 | -0.1281167 | 0.2216979 | 0.6000539 | 0.3903602 |
| 0.52 | 0.0460287 | 0.0883852 | -0.1272032 | 0.2192606 | 0.6025246 | 0.3907663 |
| 0.53 | 0.0452585 | 0.0875206 | -0.1262787 | 0.2167957 | 0.6050736 | 0.3911689 |
| 0.54 | 0.0444797 | 0.0866458 | -0.1253430 | 0.2143024 | 0.6077058 | 0.3915679 |
| 0.55 | 0.0436920 | 0.0857606 | -0.1243956 | 0.2117797 | 0.6104262 | 0.3919630 |
| 0.56 | 0.0428951 | 0.0848646 | -0.1234364 | 0.2092266 | 0.6132404 | 0.3923540 |
| 0.57 | 0.0420885 | 0.0839574 | -0.1224649 | 0.2066419 | 0.6161542 | 0.3927409 |
| 0.58 | 0.0412719 | 0.0830386 | -0.1214808 | 0.2040247 | 0.6191743 | 0.3931233 |
| 0.59 | 0.0404449 | 0.0821079 | -0.1204837 | 0.2013735 | 0.6223076 | 0.3935011 |
| 0.60 | 0.0396071 | 0.0811649 | -0.1194731 | 0.1986873 | 0.6255618 | 0.3938740 |
| 0.61 | 0.0387579 | 0.0802089 | -0.1184488 | 0.1959645 | 0.6289455 | 0.3942417 |
| 0.62 | 0.0378969 | 0.0792397 | -0.1174101 | 0.1932039 | 0.6324676 | 0.3946041 |
| 0.63 | 0.0370236 | 0.0782566 | -0.1163566 | 0.1904038 | 0.6361383 | 0.3949607 |
| 0.64 | 0.0361374 | 0.0772591 | -0.1152877 | 0.1875625 | 0.6399687 | 0.3953114 |
| 0.65 | 0.0352377 | 0.0762467 | -0.1142030 | 0.1846784 | 0.6439708 | 0.3956559 |
| 0.66 | 0.0343239 | 0.0752185 | -0.1131017 | 0.1817496 | 0.6481580 | 0.3959936 |
| 0.67 | 0.0333953 | 0.0741741 | -0.1119833 | 0.1787739 | 0.6525453 | 0.3963244 |
| 0.68 | 0.0324513 | 0.0731126 | -0.1108469 | 0.1757494 | 0.6571489 | 0.3966477 |
| 0.69 | 0.0314909 | 0.0720333 | -0.1096918 | 0.1726736 | 0.6619872 | 0.3969632 |
| 0.70 | 0.0305134 | 0.0709353 | -0.1085172 | 0.1695440 | 0.6670805 | 0.3972705 |
| 0.71 | 0.0295179 | 0.0698177 | -0.1073222 | 0.1663580 | 0.6724515 | 0.3975689 |
| 0.72 | 0.0285035 | 0.0686794 | -0.1061056 | 0.1631125 | 0.6781257 | 0.3978581 |
| 0.73 | 0.0274690 | 0.0675193 | -0.1048664 | 0.1598044 | 0.6841318 | 0.3981373 |
| 0.74 | 0.0264134 | 0.0663363 | -0.1036034 | 0.1564302 | 0.6905022 | 0.3984061 |
| 0.75 | 0.0253354 | 0.0651290 | -0.1023152 | 0.1529860 | 0.6972737 | 0.3986636 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.02534 Min. :0.06513 Min. :-0.1290
1st Qu.:0.5625 1st Qu.:0.03173 1st Qu.:0.07230 1st Qu.:-0.1232
Median :0.6250 Median :0.03746 Median :0.07875 Median :-0.1169
Mean :0.6250 Mean :0.03710 Mean :0.07834 Mean :-0.1165
3rd Qu.:0.6875 3rd Qu.:0.04269 3rd Qu.:0.08464 3rd Qu.:-0.1100
Max. :0.7500 Max. :0.04754 Max. :0.09009 Max. :-0.1023
ci.ub pvalue sigma2
Min. :0.1530 Min. :0.5977 Min. :0.3900
1st Qu.:0.1734 1st Qu.:0.6140 1st Qu.:0.3925
Median :0.1918 Median :0.6343 Median :0.3948
Mean :0.1906 Mean :0.6389 Mean :0.3946
3rd Qu.:0.2086 3rd Qu.:0.6608 3rd Qu.:0.3969
Max. :0.2241 Max. :0.6973 Max. :0.3987
Numerical representation
Code
#numeric representation
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$rep_n <- as.factor(dataN33_prepost$rep_numeric)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = rep_n)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.0349218 | 0.0888484 | -0.2090615 | 0.1392179 | 0.6942832 | 0.4016268 |
| 0.51 | -0.0350664 | 0.0880240 | -0.2075903 | 0.1374575 | 0.6903552 | 0.4018744 |
| 0.52 | -0.0352110 | 0.0871894 | -0.2060992 | 0.1356772 | 0.6863269 | 0.4021171 |
| 0.53 | -0.0353556 | 0.0863444 | -0.2045875 | 0.1338763 | 0.6821934 | 0.4023546 |
| 0.54 | -0.0355002 | 0.0854885 | -0.2030546 | 0.1320542 | 0.6779495 | 0.4025868 |
| 0.55 | -0.0356448 | 0.0846215 | -0.2014999 | 0.1302103 | 0.6735897 | 0.4028134 |
| 0.56 | -0.0357895 | 0.0837430 | -0.1999227 | 0.1283438 | 0.6691080 | 0.4030342 |
| 0.57 | -0.0359341 | 0.0828526 | -0.1983223 | 0.1264540 | 0.6644979 | 0.4032489 |
| 0.58 | -0.0360789 | 0.0819500 | -0.1966979 | 0.1245402 | 0.6597526 | 0.4034572 |
| 0.59 | -0.0362237 | 0.0810347 | -0.1950487 | 0.1226014 | 0.6548647 | 0.4036590 |
| 0.60 | -0.0363685 | 0.0801062 | -0.1933739 | 0.1206368 | 0.6498262 | 0.4038538 |
| 0.61 | -0.0365135 | 0.0791642 | -0.1916725 | 0.1186455 | 0.6446285 | 0.4040413 |
| 0.62 | -0.0366585 | 0.0782081 | -0.1899435 | 0.1166265 | 0.6392623 | 0.4042213 |
| 0.63 | -0.0368036 | 0.0772373 | -0.1881860 | 0.1145787 | 0.6337176 | 0.4043934 |
| 0.64 | -0.0369489 | 0.0762514 | -0.1863988 | 0.1125010 | 0.6279833 | 0.4045571 |
| 0.65 | -0.0370943 | 0.0752496 | -0.1845808 | 0.1103922 | 0.6220477 | 0.4047121 |
| 0.66 | -0.0372398 | 0.0742314 | -0.1827307 | 0.1082510 | 0.6158979 | 0.4048580 |
| 0.67 | -0.0373855 | 0.0731960 | -0.1808471 | 0.1060760 | 0.6095196 | 0.4049942 |
| 0.68 | -0.0375314 | 0.0721427 | -0.1789285 | 0.1038657 | 0.6028976 | 0.4051202 |
| 0.69 | -0.0376774 | 0.0710707 | -0.1769735 | 0.1016187 | 0.5960149 | 0.4052356 |
| 0.70 | -0.0378237 | 0.0699792 | -0.1749804 | 0.0993330 | 0.5888529 | 0.4053397 |
| 0.71 | -0.0379701 | 0.0688671 | -0.1729471 | 0.0970069 | 0.5813913 | 0.4054320 |
| 0.72 | -0.0381168 | 0.0677335 | -0.1708719 | 0.0946384 | 0.5736075 | 0.4055117 |
| 0.73 | -0.0382636 | 0.0665772 | -0.1687525 | 0.0922252 | 0.5654766 | 0.4055781 |
| 0.74 | -0.0384107 | 0.0653970 | -0.1665865 | 0.0897651 | 0.5569708 | 0.4056305 |
| 0.75 | -0.0385580 | 0.0641916 | -0.1643714 | 0.0872553 | 0.5480592 | 0.4056679 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.03856 Min. :0.06419 Min. :-0.2091
1st Qu.:0.5625 1st Qu.:-0.03764 1st Qu.:0.07134 1st Qu.:-0.1995
Median :0.6250 Median :-0.03673 Median :0.07772 Median :-0.1891
Mean :0.6250 Mean :-0.03673 Mean :0.07730 Mean :-0.1882
3rd Qu.:0.6875 3rd Qu.:-0.03583 3rd Qu.:0.08352 3rd Qu.:-0.1775
Max. :0.7500 Max. :-0.03492 Max. :0.08885 Max. :-0.1644
ci.ub pvalue sigma2
Min. :0.08726 Min. :0.5481 Min. :0.4016
1st Qu.:0.10218 1st Qu.:0.5977 1st Qu.:0.4031
Median :0.11560 Median :0.6365 Median :0.4043
Mean :0.11476 Mean :0.6311 Mean :0.4041
3rd Qu.:0.12787 3rd Qu.:0.6680 3rd Qu.:0.4052
Max. :0.13922 Max. :0.6943 Max. :0.4057
Highlighting representation
Code
#highlighting representation
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$rep_h <- as.factor(dataN33_prepost$rep_highlighting)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = rep_h)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.0291142 | 0.1151879 | -0.1966498 | 0.2548783 | 0.8004581 | 0.3963487 |
| 0.51 | 0.0283114 | 0.1141681 | -0.1954540 | 0.2520769 | 0.8041497 | 0.3966994 |
| 0.52 | 0.0274993 | 0.1131361 | -0.1942433 | 0.2492420 | 0.8079556 | 0.3970460 |
| 0.53 | 0.0266776 | 0.1120914 | -0.1930174 | 0.2463727 | 0.8118819 | 0.3973886 |
| 0.54 | 0.0258460 | 0.1110335 | -0.1917757 | 0.2434677 | 0.8159351 | 0.3977267 |
| 0.55 | 0.0250041 | 0.1099622 | -0.1905179 | 0.2405260 | 0.8201223 | 0.3980603 |
| 0.56 | 0.0241515 | 0.1088769 | -0.1892434 | 0.2375464 | 0.8244508 | 0.3983892 |
| 0.57 | 0.0232880 | 0.1077773 | -0.1879516 | 0.2345277 | 0.8289289 | 0.3987131 |
| 0.58 | 0.0224132 | 0.1066628 | -0.1866421 | 0.2314685 | 0.8335651 | 0.3990318 |
| 0.59 | 0.0215267 | 0.1055330 | -0.1853142 | 0.2283675 | 0.8383688 | 0.3993450 |
| 0.60 | 0.0206280 | 0.1043872 | -0.1839672 | 0.2252232 | 0.8433500 | 0.3996526 |
| 0.61 | 0.0197167 | 0.1032250 | -0.1826005 | 0.2220340 | 0.8485197 | 0.3999541 |
| 0.62 | 0.0187925 | 0.1020457 | -0.1812134 | 0.2187984 | 0.8538897 | 0.4002493 |
| 0.63 | 0.0178548 | 0.1008487 | -0.1798049 | 0.2155146 | 0.8594726 | 0.4005379 |
| 0.64 | 0.0169032 | 0.0996332 | -0.1783744 | 0.2121807 | 0.8652823 | 0.4008195 |
| 0.65 | 0.0159371 | 0.0983986 | -0.1769207 | 0.2087948 | 0.8713339 | 0.4010937 |
| 0.66 | 0.0149560 | 0.0971441 | -0.1754429 | 0.2053549 | 0.8776438 | 0.4013602 |
| 0.67 | 0.0139594 | 0.0958687 | -0.1739398 | 0.2018586 | 0.8842298 | 0.4016186 |
| 0.68 | 0.0129466 | 0.0945717 | -0.1724104 | 0.1983037 | 0.8911116 | 0.4018683 |
| 0.69 | 0.0119172 | 0.0932519 | -0.1708532 | 0.1946876 | 0.8983106 | 0.4021089 |
| 0.70 | 0.0108704 | 0.0919084 | -0.1692667 | 0.1910075 | 0.9058503 | 0.4023398 |
| 0.71 | 0.0098056 | 0.0905400 | -0.1676495 | 0.1872607 | 0.9137566 | 0.4025605 |
| 0.72 | 0.0087221 | 0.0891454 | -0.1659996 | 0.1834439 | 0.9220581 | 0.4027703 |
| 0.73 | 0.0076192 | 0.0877233 | -0.1643153 | 0.1795537 | 0.9307866 | 0.4029685 |
| 0.74 | 0.0064962 | 0.0862722 | -0.1625942 | 0.1755866 | 0.9399771 | 0.4031545 |
| 0.75 | 0.0053522 | 0.0847905 | -0.1608340 | 0.1715385 | 0.9496686 | 0.4033273 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.005352 Min. :0.08479 Min. :-0.1966
1st Qu.:0.5625 1st Qu.:0.012175 1st Qu.:0.09358 1st Qu.:-0.1889
Median :0.6250 Median :0.018324 Median :0.10145 Median :-0.1805
Mean :0.6250 Mean :0.017935 Mean :0.10093 Mean :-0.1799
3rd Qu.:0.6875 3rd Qu.:0.023936 3rd Qu.:0.10860 3rd Qu.:-0.1712
Max. :0.7500 Max. :0.029114 Max. :0.11519 Max. :-0.1608
ci.ub pvalue sigma2
Min. :0.1715 Min. :0.8005 Min. :0.3963
1st Qu.:0.1956 1st Qu.:0.8256 1st Qu.:0.3985
Median :0.2172 Median :0.8567 Median :0.4004
Mean :0.2158 Mean :0.8631 Mean :0.4002
3rd Qu.:0.2368 3rd Qu.:0.8965 3rd Qu.:0.4020
Max. :0.2549 Max. :0.9497 Max. :0.4033
Text-based representation
Code
#text-based representation
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$rep_t <- as.factor(dataN33_prepost$rep_text_based)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = rep_t)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.0504044 | 0.0931938 | -0.1322521 | 0.2330608 | 0.5886071 | 0.4074751 |
| 0.51 | 0.0507746 | 0.0923394 | -0.1302072 | 0.2317564 | 0.5824087 | 0.4077671 |
| 0.52 | 0.0511455 | 0.0914743 | -0.1281408 | 0.2304317 | 0.5760769 | 0.4080540 |
| 0.53 | 0.0515169 | 0.0905982 | -0.1260522 | 0.2290861 | 0.5696065 | 0.4083356 |
| 0.54 | 0.0518890 | 0.0897107 | -0.1239408 | 0.2277188 | 0.5629920 | 0.4086117 |
| 0.55 | 0.0522616 | 0.0888116 | -0.1218060 | 0.2263292 | 0.5562272 | 0.4088819 |
| 0.56 | 0.0526348 | 0.0879005 | -0.1196469 | 0.2249166 | 0.5493061 | 0.4091462 |
| 0.57 | 0.0530086 | 0.0869769 | -0.1174629 | 0.2234802 | 0.5422219 | 0.4094041 |
| 0.58 | 0.0533830 | 0.0860405 | -0.1152532 | 0.2220192 | 0.5349676 | 0.4096555 |
| 0.59 | 0.0537580 | 0.0850907 | -0.1130168 | 0.2205327 | 0.5275357 | 0.4099000 |
| 0.60 | 0.0541335 | 0.0841273 | -0.1107529 | 0.2190199 | 0.5199182 | 0.4101373 |
| 0.61 | 0.0545096 | 0.0831496 | -0.1084606 | 0.2174798 | 0.5121068 | 0.4103671 |
| 0.62 | 0.0548863 | 0.0821572 | -0.1061388 | 0.2159113 | 0.5040925 | 0.4105890 |
| 0.63 | 0.0552635 | 0.0811494 | -0.1037864 | 0.2143134 | 0.4958656 | 0.4108027 |
| 0.64 | 0.0556413 | 0.0801258 | -0.1014024 | 0.2126850 | 0.4874161 | 0.4110078 |
| 0.65 | 0.0560196 | 0.0790856 | -0.0989854 | 0.2110247 | 0.4787332 | 0.4112038 |
| 0.66 | 0.0563985 | 0.0780283 | -0.0965341 | 0.2093312 | 0.4698052 | 0.4113902 |
| 0.67 | 0.0567780 | 0.0769531 | -0.0940472 | 0.2076032 | 0.4606199 | 0.4115667 |
| 0.68 | 0.0571580 | 0.0758591 | -0.0915231 | 0.2058391 | 0.4511642 | 0.4117326 |
| 0.69 | 0.0575385 | 0.0747456 | -0.0889602 | 0.2040373 | 0.4414239 | 0.4118874 |
| 0.70 | 0.0579196 | 0.0736118 | -0.0863568 | 0.2021960 | 0.4313840 | 0.4120306 |
| 0.71 | 0.0583012 | 0.0724565 | -0.0837109 | 0.2003133 | 0.4210286 | 0.4121614 |
| 0.72 | 0.0586834 | 0.0712788 | -0.0810204 | 0.1983872 | 0.4103405 | 0.4122793 |
| 0.73 | 0.0590661 | 0.0700774 | -0.0782831 | 0.1964154 | 0.3993014 | 0.4123834 |
| 0.74 | 0.0594494 | 0.0688513 | -0.0754966 | 0.1943954 | 0.3878919 | 0.4124729 |
| 0.75 | 0.0598332 | 0.0675989 | -0.0726581 | 0.1923246 | 0.3760910 | 0.4125470 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.05040 Min. :0.06760 Min. :-0.13225
1st Qu.:0.5625 1st Qu.:0.05273 1st Qu.:0.07502 1st Qu.:-0.11910
Median :0.6250 Median :0.05507 Median :0.08165 Median :-0.10496
Mean :0.6250 Mean :0.05509 Mean :0.08121 Mean :-0.10407
3rd Qu.:0.6875 3rd Qu.:0.05744 3rd Qu.:0.08767 3rd Qu.:-0.08960
Max. :0.7500 Max. :0.05983 Max. :0.09319 Max. :-0.07266
ci.ub pvalue sigma2
Min. :0.1923 Min. :0.3761 Min. :0.4075
1st Qu.:0.2045 1st Qu.:0.4439 1st Qu.:0.4092
Median :0.2151 Median :0.5000 Median :0.4107
Mean :0.2143 Mean :0.4937 Mean :0.4105
3rd Qu.:0.2246 3rd Qu.:0.5475 3rd Qu.:0.4118
Max. :0.2331 Max. :0.5886 Max. :0.4125
Multiple representation formats
Code
#mono vs. multiple representation
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$rep_nr <- as.factor(dataN33_prepost$rep_nr)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = rep_nr)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.3057128 | 0.2115950 | -0.1090059 | 0.7204314 | 0.1485137 | 0.3646199 |
| 0.51 | 0.3039749 | 0.2107303 | -0.1090488 | 0.7169986 | 0.1491661 | 0.3649269 |
| 0.52 | 0.3021921 | 0.2098480 | -0.1091025 | 0.7134866 | 0.1498527 | 0.3652316 |
| 0.53 | 0.3003626 | 0.2089478 | -0.1091675 | 0.7098927 | 0.1505757 | 0.3655340 |
| 0.54 | 0.2984847 | 0.2080290 | -0.1092446 | 0.7062139 | 0.1513377 | 0.3658341 |
| 0.55 | 0.2965562 | 0.2070910 | -0.1093348 | 0.7024471 | 0.1521413 | 0.3661320 |
| 0.56 | 0.2945751 | 0.2061333 | -0.1094388 | 0.6985890 | 0.1529895 | 0.3664276 |
| 0.57 | 0.2925392 | 0.2051553 | -0.1095577 | 0.6946362 | 0.1538855 | 0.3667210 |
| 0.58 | 0.2904462 | 0.2041561 | -0.1096924 | 0.6905848 | 0.1548327 | 0.3670122 |
| 0.59 | 0.2882936 | 0.2031352 | -0.1098441 | 0.6864312 | 0.1558348 | 0.3673012 |
| 0.60 | 0.2860787 | 0.2020917 | -0.1100138 | 0.6821713 | 0.1568960 | 0.3675882 |
| 0.61 | 0.2837989 | 0.2010250 | -0.1102028 | 0.6778006 | 0.1580208 | 0.3678731 |
| 0.62 | 0.2814511 | 0.1999341 | -0.1104125 | 0.6733147 | 0.1592141 | 0.3681560 |
| 0.63 | 0.2790323 | 0.1988182 | -0.1106441 | 0.6687088 | 0.1604812 | 0.3684371 |
| 0.64 | 0.2765392 | 0.1976764 | -0.1108993 | 0.6639777 | 0.1618282 | 0.3687164 |
| 0.65 | 0.2739682 | 0.1965076 | -0.1111797 | 0.6591161 | 0.1632614 | 0.3689941 |
| 0.66 | 0.2713156 | 0.1953110 | -0.1114869 | 0.6541180 | 0.1647882 | 0.3692702 |
| 0.67 | 0.2685774 | 0.1940853 | -0.1118228 | 0.6489775 | 0.1664163 | 0.3695450 |
| 0.68 | 0.2657493 | 0.1928294 | -0.1121894 | 0.6436880 | 0.1681547 | 0.3698186 |
| 0.69 | 0.2628269 | 0.1915422 | -0.1125889 | 0.6382426 | 0.1700130 | 0.3700912 |
| 0.70 | 0.2598052 | 0.1902222 | -0.1130234 | 0.6326338 | 0.1720022 | 0.3703631 |
| 0.71 | 0.2566791 | 0.1888681 | -0.1134955 | 0.6268537 | 0.1741342 | 0.3706346 |
| 0.72 | 0.2534430 | 0.1874783 | -0.1140078 | 0.6208938 | 0.1764226 | 0.3709059 |
| 0.73 | 0.2500910 | 0.1860514 | -0.1145631 | 0.6147451 | 0.1788826 | 0.3711774 |
| 0.74 | 0.2466167 | 0.1845856 | -0.1151645 | 0.6083978 | 0.1815311 | 0.3714496 |
| 0.75 | 0.2430132 | 0.1830791 | -0.1158153 | 0.6018416 | 0.1843873 | 0.3717228 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.2430 Min. :0.1831 Min. :-0.1158
1st Qu.:0.5625 1st Qu.:0.2636 1st Qu.:0.1919 1st Qu.:-0.1125
Median :0.6250 Median :0.2802 Median :0.1994 Median :-0.1105
Mean :0.6250 Mean :0.2782 Mean :0.1987 Mean :-0.1112
3rd Qu.:0.6875 3rd Qu.:0.2941 3rd Qu.:0.2059 3rd Qu.:-0.1095
Max. :0.7500 Max. :0.3057 Max. :0.2116 Max. :-0.1090
ci.ub pvalue sigma2
Min. :0.6018 Min. :0.1485 Min. :0.3646
1st Qu.:0.6396 1st Qu.:0.1532 1st Qu.:0.3665
Median :0.6710 Median :0.1598 Median :0.3683
Mean :0.6675 Mean :0.1621 Mean :0.3682
3rd Qu.:0.6976 3rd Qu.:0.1695 3rd Qu.:0.3700
Max. :0.7204 Max. :0.1844 Max. :0.3717
Level of feedback
Code
## LEVEL ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$order<- as.factor(dataN33_prepost$FB_order)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = order)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.2663898 | 0.3260957 | -0.3727460 | 0.9055255 | 0.4139817 | 0.4149546 |
| 0.51 | 0.2663955 | 0.3261140 | -0.3727761 | 0.9055671 | 0.4139978 | 0.4152738 |
| 0.52 | 0.2664048 | 0.3261300 | -0.3727982 | 0.9056078 | 0.4140045 | 0.4155871 |
| 0.53 | 0.2664176 | 0.3261437 | -0.3728122 | 0.9056475 | 0.4140016 | 0.4158945 |
| 0.54 | 0.2664342 | 0.3261549 | -0.3728176 | 0.9056861 | 0.4139886 | 0.4161955 |
| 0.55 | 0.2664548 | 0.3261636 | -0.3728142 | 0.9057237 | 0.4139651 | 0.4164901 |
| 0.56 | 0.2664793 | 0.3261697 | -0.3728016 | 0.9057602 | 0.4139307 | 0.4167779 |
| 0.57 | 0.2665082 | 0.3261730 | -0.3727793 | 0.9057956 | 0.4138850 | 0.4170586 |
| 0.58 | 0.2665414 | 0.3261735 | -0.3727469 | 0.9058297 | 0.4138275 | 0.4173318 |
| 0.59 | 0.2665792 | 0.3261710 | -0.3727042 | 0.9058626 | 0.4137576 | 0.4175974 |
| 0.60 | 0.2666218 | 0.3261653 | -0.3726505 | 0.9058941 | 0.4136749 | 0.4178548 |
| 0.61 | 0.2666695 | 0.3261564 | -0.3725854 | 0.9059243 | 0.4135787 | 0.4181038 |
| 0.62 | 0.2667224 | 0.3261441 | -0.3725083 | 0.9059530 | 0.4134684 | 0.4183440 |
| 0.63 | 0.2667808 | 0.3261282 | -0.3724187 | 0.9059802 | 0.4133433 | 0.4185748 |
| 0.64 | 0.2668449 | 0.3261085 | -0.3723160 | 0.9060059 | 0.4132029 | 0.4187959 |
| 0.65 | 0.2669151 | 0.3260849 | -0.3721996 | 0.9060299 | 0.4130462 | 0.4190069 |
| 0.66 | 0.2669917 | 0.3260573 | -0.3720688 | 0.9060522 | 0.4128725 | 0.4192071 |
| 0.67 | 0.2670750 | 0.3260252 | -0.3719227 | 0.9060727 | 0.4126809 | 0.4193960 |
| 0.68 | 0.2671653 | 0.3259886 | -0.3717607 | 0.9060913 | 0.4124704 | 0.4195730 |
| 0.69 | 0.2672631 | 0.3259473 | -0.3715818 | 0.9061080 | 0.4122400 | 0.4197375 |
| 0.70 | 0.2673687 | 0.3259008 | -0.3713851 | 0.9061226 | 0.4119886 | 0.4198889 |
| 0.71 | 0.2674827 | 0.3258490 | -0.3711697 | 0.9061351 | 0.4117150 | 0.4200263 |
| 0.72 | 0.2676056 | 0.3257916 | -0.3709343 | 0.9061455 | 0.4114178 | 0.4201490 |
| 0.73 | 0.2677379 | 0.3257282 | -0.3706777 | 0.9061536 | 0.4110956 | 0.4202560 |
| 0.74 | 0.2678803 | 0.3256585 | -0.3703987 | 0.9061593 | 0.4107467 | 0.4203466 |
| 0.75 | 0.2680335 | 0.3255821 | -0.3700957 | 0.9061627 | 0.4103693 | 0.4204195 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.2664 Min. :0.3256 Min. :-0.3728
1st Qu.:0.5625 1st Qu.:0.2665 1st Qu.:0.3260 1st Qu.:-0.3728
Median :0.6250 Median :0.2668 Median :0.3261 Median :-0.3725
Mean :0.6250 Mean :0.2669 Mean :0.3260 Mean :-0.3721
3rd Qu.:0.6875 3rd Qu.:0.2672 3rd Qu.:0.3262 3rd Qu.:-0.3716
Max. :0.7500 Max. :0.2680 Max. :0.3262 Max. :-0.3701
ci.ub pvalue sigma2
Min. :0.9055 Min. :0.4104 Min. :0.4150
1st Qu.:0.9058 1st Qu.:0.4123 1st Qu.:0.4168
Median :0.9060 Median :0.4134 Median :0.4185
Mean :0.9059 Mean :0.4130 Mean :0.4182
3rd Qu.:0.9061 3rd Qu.:0.4139 3rd Qu.:0.4197
Max. :0.9062 Max. :0.4140 Max. :0.4204
Higher level only
Code
## HIGHER LEVEL ONLY ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$order_hi <- as.factor(ifelse(dataN33_prepost$FB_order == 2, 1, 0))
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = order_hi)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.2663898 | 0.3260957 | -0.9055256 | 0.3727460 | 0.4139817 | 0.4149546 |
| 0.51 | -0.2663955 | 0.3261140 | -0.9055671 | 0.3727761 | 0.4139978 | 0.4152738 |
| 0.52 | -0.2664048 | 0.3261300 | -0.9056078 | 0.3727982 | 0.4140045 | 0.4155871 |
| 0.53 | -0.2664176 | 0.3261437 | -0.9056475 | 0.3728122 | 0.4140016 | 0.4158945 |
| 0.54 | -0.2664343 | 0.3261549 | -0.9056862 | 0.3728177 | 0.4139886 | 0.4161955 |
| 0.55 | -0.2664548 | 0.3261636 | -0.9057237 | 0.3728142 | 0.4139651 | 0.4164901 |
| 0.56 | -0.2664793 | 0.3261697 | -0.9057602 | 0.3728016 | 0.4139307 | 0.4167779 |
| 0.57 | -0.2665082 | 0.3261730 | -0.9057956 | 0.3727793 | 0.4138850 | 0.4170586 |
| 0.58 | -0.2665414 | 0.3261735 | -0.9058297 | 0.3727469 | 0.4138275 | 0.4173318 |
| 0.59 | -0.2665792 | 0.3261710 | -0.9058626 | 0.3727042 | 0.4137576 | 0.4175974 |
| 0.60 | -0.2666218 | 0.3261653 | -0.9058941 | 0.3726505 | 0.4136749 | 0.4178548 |
| 0.61 | -0.2666695 | 0.3261564 | -0.9059243 | 0.3725853 | 0.4135786 | 0.4181038 |
| 0.62 | -0.2667224 | 0.3261441 | -0.9059530 | 0.3725083 | 0.4134684 | 0.4183440 |
| 0.63 | -0.2667808 | 0.3261282 | -0.9059802 | 0.3724187 | 0.4133433 | 0.4185748 |
| 0.64 | -0.2668449 | 0.3261085 | -0.9060059 | 0.3723160 | 0.4132029 | 0.4187960 |
| 0.65 | -0.2669151 | 0.3260849 | -0.9060299 | 0.3721996 | 0.4130462 | 0.4190069 |
| 0.66 | -0.2669917 | 0.3260572 | -0.9060522 | 0.3720688 | 0.4128725 | 0.4192070 |
| 0.67 | -0.2670750 | 0.3260252 | -0.9060727 | 0.3719227 | 0.4126809 | 0.4193960 |
| 0.68 | -0.2671653 | 0.3259886 | -0.9060913 | 0.3717607 | 0.4124704 | 0.4195730 |
| 0.69 | -0.2672631 | 0.3259473 | -0.9061080 | 0.3715818 | 0.4122400 | 0.4197375 |
| 0.70 | -0.2673687 | 0.3259008 | -0.9061226 | 0.3713851 | 0.4119886 | 0.4198889 |
| 0.71 | -0.2674827 | 0.3258491 | -0.9061351 | 0.3711697 | 0.4117150 | 0.4200263 |
| 0.72 | -0.2676056 | 0.3257916 | -0.9061455 | 0.3709342 | 0.4114178 | 0.4201489 |
| 0.73 | -0.2677379 | 0.3257282 | -0.9061536 | 0.3706777 | 0.4110956 | 0.4202560 |
| 0.74 | -0.2678803 | 0.3256585 | -0.9061593 | 0.3703987 | 0.4107467 | 0.4203465 |
| 0.75 | -0.2680335 | 0.3255821 | -0.9061627 | 0.3700957 | 0.4103693 | 0.4204195 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.2680 Min. :0.3256 Min. :-0.9062
1st Qu.:0.5625 1st Qu.:-0.2672 1st Qu.:0.3260 1st Qu.:-0.9061
Median :0.6250 Median :-0.2668 Median :0.3261 Median :-0.9060
Mean :0.6250 Mean :-0.2669 Mean :0.3260 Mean :-0.9059
3rd Qu.:0.6875 3rd Qu.:-0.2665 3rd Qu.:0.3262 3rd Qu.:-0.9058
Max. :0.7500 Max. :-0.2664 Max. :0.3262 Max. :-0.9055
ci.ub pvalue sigma2
Min. :0.3701 Min. :0.4104 Min. :0.4150
1st Qu.:0.3716 1st Qu.:0.4123 1st Qu.:0.4168
Median :0.3725 Median :0.4134 Median :0.4185
Mean :0.3721 Mean :0.4130 Mean :0.4182
3rd Qu.:0.3728 3rd Qu.:0.4139 3rd Qu.:0.4197
Max. :0.3728 Max. :0.4140 Max. :0.4204
####Both lower and higher level
Code
## BOTH LOWER AND HIGHER LEVEL ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$order_hilow <- ifelse(dataN33_prepost$FB_order == 3, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = order_hilow)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.2663898 | 0.3260957 | -0.3727460 | 0.9055255 | 0.4139817 | 0.4149546 |
| 0.51 | 0.2663955 | 0.3261140 | -0.3727761 | 0.9055671 | 0.4139978 | 0.4152738 |
| 0.52 | 0.2664048 | 0.3261300 | -0.3727982 | 0.9056078 | 0.4140045 | 0.4155871 |
| 0.53 | 0.2664176 | 0.3261437 | -0.3728122 | 0.9056475 | 0.4140016 | 0.4158945 |
| 0.54 | 0.2664342 | 0.3261549 | -0.3728176 | 0.9056861 | 0.4139886 | 0.4161955 |
| 0.55 | 0.2664548 | 0.3261636 | -0.3728142 | 0.9057237 | 0.4139651 | 0.4164901 |
| 0.56 | 0.2664793 | 0.3261697 | -0.3728016 | 0.9057602 | 0.4139307 | 0.4167779 |
| 0.57 | 0.2665082 | 0.3261730 | -0.3727793 | 0.9057956 | 0.4138850 | 0.4170586 |
| 0.58 | 0.2665414 | 0.3261735 | -0.3727469 | 0.9058297 | 0.4138275 | 0.4173318 |
| 0.59 | 0.2665792 | 0.3261710 | -0.3727042 | 0.9058626 | 0.4137576 | 0.4175974 |
| 0.60 | 0.2666218 | 0.3261653 | -0.3726505 | 0.9058941 | 0.4136749 | 0.4178548 |
| 0.61 | 0.2666695 | 0.3261564 | -0.3725853 | 0.9059243 | 0.4135786 | 0.4181038 |
| 0.62 | 0.2667224 | 0.3261441 | -0.3725083 | 0.9059530 | 0.4134684 | 0.4183440 |
| 0.63 | 0.2667808 | 0.3261282 | -0.3724187 | 0.9059802 | 0.4133433 | 0.4185748 |
| 0.64 | 0.2668449 | 0.3261085 | -0.3723160 | 0.9060059 | 0.4132029 | 0.4187959 |
| 0.65 | 0.2669151 | 0.3260849 | -0.3721996 | 0.9060299 | 0.4130462 | 0.4190069 |
| 0.66 | 0.2669917 | 0.3260572 | -0.3720688 | 0.9060522 | 0.4128725 | 0.4192070 |
| 0.67 | 0.2670750 | 0.3260252 | -0.3719227 | 0.9060727 | 0.4126809 | 0.4193960 |
| 0.68 | 0.2671653 | 0.3259886 | -0.3717607 | 0.9060913 | 0.4124704 | 0.4195730 |
| 0.69 | 0.2672631 | 0.3259473 | -0.3715818 | 0.9061080 | 0.4122400 | 0.4197375 |
| 0.70 | 0.2673687 | 0.3259008 | -0.3713851 | 0.9061226 | 0.4119886 | 0.4198889 |
| 0.71 | 0.2674827 | 0.3258490 | -0.3711697 | 0.9061351 | 0.4117150 | 0.4200263 |
| 0.72 | 0.2676056 | 0.3257916 | -0.3709343 | 0.9061455 | 0.4114178 | 0.4201489 |
| 0.73 | 0.2677379 | 0.3257282 | -0.3706777 | 0.9061536 | 0.4110956 | 0.4202560 |
| 0.74 | 0.2678803 | 0.3256585 | -0.3703987 | 0.9061593 | 0.4107467 | 0.4203465 |
| 0.75 | 0.2680335 | 0.3255821 | -0.3700957 | 0.9061627 | 0.4103693 | 0.4204195 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.2664 Min. :0.3256 Min. :-0.3728
1st Qu.:0.5625 1st Qu.:0.2665 1st Qu.:0.3260 1st Qu.:-0.3728
Median :0.6250 Median :0.2668 Median :0.3261 Median :-0.3725
Mean :0.6250 Mean :0.2669 Mean :0.3260 Mean :-0.3721
3rd Qu.:0.6875 3rd Qu.:0.2672 3rd Qu.:0.3262 3rd Qu.:-0.3716
Max. :0.7500 Max. :0.2680 Max. :0.3262 Max. :-0.3701
ci.ub pvalue sigma2
Min. :0.9055 Min. :0.4104 Min. :0.4150
1st Qu.:0.9058 1st Qu.:0.4123 1st Qu.:0.4168
Median :0.9060 Median :0.4134 Median :0.4185
Mean :0.9059 Mean :0.4130 Mean :0.4182
3rd Qu.:0.9061 3rd Qu.:0.4139 3rd Qu.:0.4197
Max. :0.9062 Max. :0.4140 Max. :0.4204
Specificity
Code
## SPECIFICITY ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$spec <- as.factor(dataN33_prepost$FB_specificity)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = spec)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.3986597 | 0.4950525 | -0.5716252 | 1.368945 | 0.4206536 | 0.4138367 |
| 0.51 | 0.3985402 | 0.4950218 | -0.5716848 | 1.368765 | 0.4207641 | 0.4141115 |
| 0.52 | 0.3984099 | 0.4949888 | -0.5717502 | 1.368570 | 0.4208850 | 0.4143823 |
| 0.53 | 0.3982683 | 0.4949531 | -0.5718220 | 1.368359 | 0.4210167 | 0.4146487 |
| 0.54 | 0.3981147 | 0.4949148 | -0.5719005 | 1.368130 | 0.4211599 | 0.4149107 |
| 0.55 | 0.3979486 | 0.4948737 | -0.5719861 | 1.367883 | 0.4213151 | 0.4151680 |
| 0.56 | 0.3977694 | 0.4948301 | -0.5720797 | 1.367619 | 0.4214833 | 0.4154209 |
| 0.57 | 0.3975762 | 0.4947831 | -0.5721808 | 1.367333 | 0.4216648 | 0.4156683 |
| 0.58 | 0.3973684 | 0.4947330 | -0.5722905 | 1.367027 | 0.4218605 | 0.4159105 |
| 0.59 | 0.3971451 | 0.4946797 | -0.5724093 | 1.366700 | 0.4220713 | 0.4161472 |
| 0.60 | 0.3969055 | 0.4946230 | -0.5725378 | 1.366349 | 0.4222982 | 0.4163782 |
| 0.61 | 0.3966486 | 0.4945628 | -0.5726766 | 1.365974 | 0.4225421 | 0.4166033 |
| 0.62 | 0.3963734 | 0.4944989 | -0.5728266 | 1.365573 | 0.4228042 | 0.4168221 |
| 0.63 | 0.3960787 | 0.4944311 | -0.5729885 | 1.365146 | 0.4230856 | 0.4170345 |
| 0.64 | 0.3957633 | 0.4943594 | -0.5731633 | 1.364690 | 0.4233877 | 0.4172401 |
| 0.65 | 0.3954259 | 0.4942834 | -0.5733518 | 1.364204 | 0.4237118 | 0.4174386 |
| 0.66 | 0.3950651 | 0.4942031 | -0.5735552 | 1.363685 | 0.4240595 | 0.4176297 |
| 0.67 | 0.3946792 | 0.4941182 | -0.5737747 | 1.363133 | 0.4244327 | 0.4178130 |
| 0.68 | 0.3942666 | 0.4940286 | -0.5740116 | 1.362545 | 0.4248331 | 0.4179883 |
| 0.69 | 0.3938253 | 0.4939339 | -0.5742673 | 1.361918 | 0.4252629 | 0.4181550 |
| 0.70 | 0.3933532 | 0.4938339 | -0.5745435 | 1.361250 | 0.4257244 | 0.4183128 |
| 0.71 | 0.3928480 | 0.4937284 | -0.5748420 | 1.360538 | 0.4262202 | 0.4184613 |
| 0.72 | 0.3923071 | 0.4936172 | -0.5751648 | 1.359779 | 0.4267533 | 0.4186000 |
| 0.73 | 0.3917275 | 0.4934999 | -0.5755144 | 1.358969 | 0.4273268 | 0.4187283 |
| 0.74 | 0.3911062 | 0.4933761 | -0.5758932 | 1.358106 | 0.4279444 | 0.4188459 |
| 0.75 | 0.3904395 | 0.4932457 | -0.5763043 | 1.357183 | 0.4286102 | 0.4189520 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.3904 Min. :0.4932 Min. :-0.5763
1st Qu.:0.5625 1st Qu.:0.3939 1st Qu.:0.4940 1st Qu.:-0.5742
Median :0.6250 Median :0.3962 Median :0.4945 Median :-0.5729
Mean :0.6250 Mean :0.3956 Mean :0.4944 Mean :-0.5733
3rd Qu.:0.6875 3rd Qu.:0.3977 3rd Qu.:0.4948 3rd Qu.:-0.5721
Max. :0.7500 Max. :0.3987 Max. :0.4951 Max. :-0.5716
ci.ub pvalue sigma2
Min. :1.357 Min. :0.4207 Min. :0.4138
1st Qu.:1.362 1st Qu.:0.4215 1st Qu.:0.4155
Median :1.365 Median :0.4229 Median :0.4169
Mean :1.365 Mean :0.4235 Mean :0.4167
3rd Qu.:1.368 3rd Qu.:0.4252 3rd Qu.:0.4181
Max. :1.369 Max. :0.4286 Max. :0.4190
Tool numbers
Code
## TOOL NUMBERS ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$tool <- as.factor(dataN33_prepost$FB_tool_numbers)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = tool)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.1242885 | 0.0637999 | -0.2493340 | 0.0007571 | 0.0514033 | 0.3321570 |
| 0.51 | -0.1242588 | 0.0638123 | -0.2493286 | 0.0008111 | 0.0515043 | 0.3325097 |
| 0.52 | -0.1242277 | 0.0638244 | -0.2493212 | 0.0008658 | 0.0516069 | 0.3328590 |
| 0.53 | -0.1241952 | 0.0638361 | -0.2493117 | 0.0009213 | 0.0517110 | 0.3332048 |
| 0.54 | -0.1241613 | 0.0638475 | -0.2493000 | 0.0009775 | 0.0518166 | 0.3335469 |
| 0.55 | -0.1241258 | 0.0638584 | -0.2492860 | 0.0010344 | 0.0519238 | 0.3338851 |
| 0.56 | -0.1240888 | 0.0638690 | -0.2492697 | 0.0010922 | 0.0520326 | 0.3342192 |
| 0.57 | -0.1240502 | 0.0638791 | -0.2492510 | 0.0011507 | 0.0521431 | 0.3345490 |
| 0.58 | -0.1240099 | 0.0638889 | -0.2492297 | 0.0012100 | 0.0522552 | 0.3348744 |
| 0.59 | -0.1239678 | 0.0638981 | -0.2492058 | 0.0012701 | 0.0523692 | 0.3351949 |
| 0.60 | -0.1239240 | 0.0639069 | -0.2491792 | 0.0013311 | 0.0524849 | 0.3355105 |
| 0.61 | -0.1238784 | 0.0639151 | -0.2491496 | 0.0013929 | 0.0526024 | 0.3358208 |
| 0.62 | -0.1238308 | 0.0639228 | -0.2491172 | 0.0014556 | 0.0527218 | 0.3361255 |
| 0.63 | -0.1237812 | 0.0639299 | -0.2490816 | 0.0015191 | 0.0528430 | 0.3364243 |
| 0.64 | -0.1237296 | 0.0639365 | -0.2490428 | 0.0015836 | 0.0529662 | 0.3367169 |
| 0.65 | -0.1236759 | 0.0639424 | -0.2490006 | 0.0016489 | 0.0530914 | 0.3370029 |
| 0.66 | -0.1236199 | 0.0639476 | -0.2489549 | 0.0017151 | 0.0532185 | 0.3372819 |
| 0.67 | -0.1235617 | 0.0639521 | -0.2489055 | 0.0017822 | 0.0533476 | 0.3375535 |
| 0.68 | -0.1235011 | 0.0639559 | -0.2488523 | 0.0018501 | 0.0534786 | 0.3378172 |
| 0.69 | -0.1234380 | 0.0639588 | -0.2487950 | 0.0019190 | 0.0536117 | 0.3380725 |
| 0.70 | -0.1233724 | 0.0639609 | -0.2487336 | 0.0019887 | 0.0537468 | 0.3383188 |
| 0.71 | -0.1233042 | 0.0639621 | -0.2486677 | 0.0020593 | 0.0538839 | 0.3385556 |
| 0.72 | -0.1232333 | 0.0639624 | -0.2485973 | 0.0021306 | 0.0540228 | 0.3387822 |
| 0.73 | -0.1231596 | 0.0639616 | -0.2485220 | 0.0022027 | 0.0541636 | 0.3389979 |
| 0.74 | -0.1230831 | 0.0639597 | -0.2484417 | 0.0022756 | 0.0543062 | 0.3392019 |
| 0.75 | -0.1230035 | 0.0639565 | -0.2483560 | 0.0023490 | 0.0544504 | 0.3393933 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.1243 Min. :0.06380 Min. :-0.2493
1st Qu.:0.5625 1st Qu.:-0.1241 1st Qu.:0.06387 1st Qu.:-0.2493
Median :0.6250 Median :-0.1238 Median :0.06393 Median :-0.2491
Mean :0.6250 Mean :-0.1237 Mean :0.06391 Mean :-0.2490
3rd Qu.:0.6875 3rd Qu.:-0.1235 3rd Qu.:0.06396 3rd Qu.:-0.2488
Max. :0.7500 Max. :-0.1230 Max. :0.06396 Max. :-0.2484
ci.ub pvalue sigma2
Min. :0.0007571 Min. :0.05140 Min. :0.3322
1st Qu.:0.0011068 1st Qu.:0.05206 1st Qu.:0.3343
Median :0.0014874 Median :0.05278 Median :0.3363
Mean :0.0015113 Mean :0.05283 Mean :0.3361
3rd Qu.:0.0019018 3rd Qu.:0.05358 3rd Qu.:0.3380
Max. :0.0023490 Max. :0.05445 Max. :0.3394
Code
#post-hoc:
CohViz <- rma(yi, vi, data=data_pp_es, subset = tool =="1")
CohViz
Random-Effects Model (k = 12; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0.2166 (SE = 0.1025)
tau (square root of estimated tau^2 value): 0.4654
I^2 (total heterogeneity / total variability): 95.09%
H^2 (total variability / sampling variability): 20.36
Test for Heterogeneity:
Q(df = 11) = 92.3366, p-val < .0001
Model Results:
estimate se zval pval ci.lb ci.ub
0.5172 0.1418 3.6481 0.0003 0.2393 0.7951 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Code
criterion <- rma(yi, vi, data=data_pp_es, subset = tool =="2")
criterion
Random-Effects Model (k = 4; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 2.8417 (SE = 2.4227)
tau (square root of estimated tau^2 value): 1.6857
I^2 (total heterogeneity / total variability): 97.59%
H^2 (total variability / sampling variability): 41.47
Test for Heterogeneity:
Q(df = 3) = 72.5270, p-val < .0001
Model Results:
estimate se zval pval ci.lb ci.ub
1.3118 0.8613 1.5231 0.1277 -0.3763 2.9999
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Code
writingPal <- rma(yi, vi, data=data_pp_es, subset = tool =="3")
writingPal
Random-Effects Model (k = 6; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0 (SE = 0.0061)
tau (square root of estimated tau^2 value): 0
I^2 (total heterogeneity / total variability): 0.00%
H^2 (total variability / sampling variability): 1.00
Test for Heterogeneity:
Q(df = 5) = 1.3625, p-val = 0.9284
Model Results:
estimate se zval pval ci.lb ci.ub
0.2042 0.0403 5.0725 <.0001 0.1253 0.2831 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Code
NCWrite <- rma(yi, vi, data=data_pp_es, subset = tool =="4")
NCWrite
Random-Effects Model (k = 2; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0.1371 (SE = 0.1978)
tau (square root of estimated tau^2 value): 0.3703
I^2 (total heterogeneity / total variability): 98.00%
H^2 (total variability / sampling variability): 50.10
Test for Heterogeneity:
Q(df = 1) = 50.1046, p-val < .0001
Model Results:
estimate se zval pval ci.lb ci.ub
0.7057 0.2645 2.6684 0.0076 0.1873 1.2240 **
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Code
eRevise <- rma(yi, vi, data=data_pp_es, subset = tool =="5")
eRevise
Random-Effects Model (k = 2; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0.0331 (SE = 0.0509)
tau (square root of estimated tau^2 value): 0.1820
I^2 (total heterogeneity / total variability): 91.98%
H^2 (total variability / sampling variability): 12.47
Test for Heterogeneity:
Q(df = 1) = 12.4698, p-val = 0.0004
Model Results:
estimate se zval pval ci.lb ci.ub
0.2825 0.1341 2.1061 0.0352 0.0196 0.5454 *
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Code
cRaterML <- rma(yi, vi, data=data_pp_es, subset = tool =="6")
cRaterML
Random-Effects Model (k = 1; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0
tau (square root of estimated tau^2 value): 0
I^2 (total heterogeneity / total variability): 0.00%
H^2 (total variability / sampling variability): 1.00
Test for Heterogeneity:
Q(df = 0) = 0.0000, p-val = 1.0000
Model Results:
estimate se zval pval ci.lb ci.ub
0.3517 0.0631 5.5714 <.0001 0.2280 0.4754 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Code
MIWrite <- rma(yi, vi, data=data_pp_es, subset = tool =="7")
MIWrite
Random-Effects Model (k = 5; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0.1333 (SE = 0.1000)
tau (square root of estimated tau^2 value): 0.3651
I^2 (total heterogeneity / total variability): 96.71%
H^2 (total variability / sampling variability): 30.38
Test for Heterogeneity:
Q(df = 4) = 69.0614, p-val < .0001
Model Results:
estimate se zval pval ci.lb ci.ub
0.3363 0.1682 1.9990 0.0456 0.0066 0.6659 *
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Code
ChatGPT <- rma(yi, vi, data=data_pp_es, subset = tool =="8")
ChatGPT
Random-Effects Model (k = 1; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0
tau (square root of estimated tau^2 value): 0
I^2 (total heterogeneity / total variability): 0.00%
H^2 (total variability / sampling variability): 1.00
Test for Heterogeneity:
Q(df = 0) = 0.0000, p-val = 1.0000
Model Results:
estimate se zval pval ci.lb ci.ub
-0.6650 0.0490 -13.5637 <.0001 -0.7611 -0.5689 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Code
dat.comp <- data.frame(estimate = c(coef(CohViz), coef(criterion), coef(writingPal), coef(NCWrite),coef(eRevise), coef(cRaterML), coef(MIWrite), coef(ChatGPT)), stderror = c(CohViz$se, criterion$se, writingPal$se, NCWrite$se, eRevise$se, cRaterML$se, MIWrite$se, ChatGPT$se), meta = c("1","2", "3", "4", "5", "6", "7", "8"), tau2 = round(c(CohViz$tau2, criterion$tau2, writingPal$tau2, NCWrite$tau2, eRevise$tau2, cRaterML$tau2, MIWrite$tau2, ChatGPT$tau2),3))
dat.comp estimate stderror meta tau2
1 0.5172196 0.14177941 1 0.217
2 1.3117891 0.86127386 2 2.842
3 0.2042270 0.04026152 3 0.000
4 0.7056804 0.26446012 4 0.137
5 0.2824778 0.13412655 5 0.033
6 0.3516930 0.06312455 6 0.000
7 0.3362509 0.16820946 7 0.133
8 -0.6649928 0.04902747 8 0.000
Code
rma(estimate, sei=stderror, mods = ~ meta, method="FE", data=dat.comp, digits=3)
Fixed-Effects with Moderators Model (k = 8)
I^2 (residual heterogeneity / unaccounted variability): 0.00%
H^2 (unaccounted variability / sampling variability): 1.00
R^2 (amount of heterogeneity accounted for): NA%
Test for Residual Heterogeneity:
QE(df = 0) = 0.000, p-val = 1.000
Test of Moderators (coefficients 2:8):
QM(df = 7) = 271.919, p-val < .001
Model Results:
estimate se zval pval ci.lb ci.ub
intrcpt 0.517 0.142 3.648 <.001 0.239 0.795 ***
meta2 0.795 0.873 0.910 0.363 -0.916 2.505
meta3 -0.313 0.147 -2.124 0.034 -0.602 -0.024 *
meta4 0.188 0.300 0.628 0.530 -0.400 0.777
meta5 -0.235 0.195 -1.203 0.229 -0.617 0.148
meta6 -0.166 0.155 -1.067 0.286 -0.470 0.139
meta7 -0.181 0.220 -0.823 0.411 -0.612 0.250
meta8 -1.182 0.150 -7.881 <.001 -1.476 -0.888 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Code
Mixed-Effects Model (k = 33; tau^2 estimator: REML)
tau^2 (estimated amount of residual heterogeneity): 0.3365 (SE = 0.1021)
tau (square root of estimated tau^2 value): 0.5801
I^2 (residual heterogeneity / unaccounted variability): 97.68%
H^2 (unaccounted variability / sampling variability): 43.04
R^2 (amount of heterogeneity accounted for): 0.00%
Test for Residual Heterogeneity:
QE(df = 25) = 297.8620, p-val < .0001
Test of Moderators (coefficients 2:8):
QM(df = 7) = 9.0746, p-val = 0.2473
Model Results:
estimate se zval pval ci.lb ci.ub
intrcpt 0.5311 0.1737 3.0585 0.0022 0.1908 0.8715 **
factor(tool)2 0.5331 0.3761 1.4173 0.1564 -0.2041 1.2703
factor(tool)3 -0.3287 0.2965 -1.1083 0.2677 -0.9099 0.2525
factor(tool)4 0.1752 0.4470 0.3920 0.6951 -0.7009 1.0513
factor(tool)5 -0.2509 0.4471 -0.5613 0.5746 -1.1271 0.6253
factor(tool)6 -0.1794 0.6088 -0.2947 0.7682 -1.3727 1.0138
factor(tool)7 -0.1855 0.3149 -0.5892 0.5558 -0.8026 0.4316
factor(tool)8 -1.1961 0.6075 -1.9688 0.0490 -2.3868 -0.0054 *
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Prior knowledge
Code
## PRIOR KNOWLEDGE ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$WQP_pre <- as.factor(dataN33_prepost$WQP_pre)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = WQP_pre)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.0161422 | 0.0049863 | -0.0259152 | -0.0063692 | 0.0012066 | 0.3961016 |
| 0.51 | -0.0161278 | 0.0049420 | -0.0258139 | -0.0064417 | 0.0011007 | 0.3964070 |
| 0.52 | -0.0161124 | 0.0048971 | -0.0257105 | -0.0065143 | 0.0010012 | 0.3967075 |
| 0.53 | -0.0160960 | 0.0048516 | -0.0256050 | -0.0065870 | 0.0009078 | 0.3970029 |
| 0.54 | -0.0160785 | 0.0048056 | -0.0254972 | -0.0066598 | 0.0008204 | 0.3972929 |
| 0.55 | -0.0160599 | 0.0047589 | -0.0253871 | -0.0067326 | 0.0007389 | 0.3975774 |
| 0.56 | -0.0160400 | 0.0047116 | -0.0252745 | -0.0068055 | 0.0006631 | 0.3978559 |
| 0.57 | -0.0160189 | 0.0046636 | -0.0251594 | -0.0068785 | 0.0005928 | 0.3981283 |
| 0.58 | -0.0159965 | 0.0046150 | -0.0250417 | -0.0069514 | 0.0005278 | 0.3983943 |
| 0.59 | -0.0159726 | 0.0045656 | -0.0249211 | -0.0070242 | 0.0004680 | 0.3986535 |
| 0.60 | -0.0159473 | 0.0045156 | -0.0247976 | -0.0070969 | 0.0004130 | 0.3989056 |
| 0.61 | -0.0159203 | 0.0044648 | -0.0246711 | -0.0071695 | 0.0003628 | 0.3991503 |
| 0.62 | -0.0158916 | 0.0044132 | -0.0245413 | -0.0072420 | 0.0003171 | 0.3993870 |
| 0.63 | -0.0158611 | 0.0043608 | -0.0244081 | -0.0073141 | 0.0002756 | 0.3996156 |
| 0.64 | -0.0158287 | 0.0043076 | -0.0242713 | -0.0073860 | 0.0002382 | 0.3998355 |
| 0.65 | -0.0157941 | 0.0042535 | -0.0241308 | -0.0074575 | 0.0002046 | 0.4000462 |
| 0.66 | -0.0157574 | 0.0041985 | -0.0239862 | -0.0075286 | 0.0001746 | 0.4002472 |
| 0.67 | -0.0157183 | 0.0041425 | -0.0238374 | -0.0075991 | 0.0001480 | 0.4004380 |
| 0.68 | -0.0156766 | 0.0040856 | -0.0236842 | -0.0076690 | 0.0001245 | 0.4006179 |
| 0.69 | -0.0156322 | 0.0040276 | -0.0235261 | -0.0077382 | 0.0001039 | 0.4007865 |
| 0.70 | -0.0155848 | 0.0039686 | -0.0233631 | -0.0078065 | 0.0000860 | 0.4009428 |
| 0.71 | -0.0155343 | 0.0039084 | -0.0231946 | -0.0078739 | 0.0000705 | 0.4010863 |
| 0.72 | -0.0154803 | 0.0038470 | -0.0230203 | -0.0079402 | 0.0000572 | 0.4012161 |
| 0.73 | -0.0154226 | 0.0037844 | -0.0228399 | -0.0080052 | 0.0000460 | 0.4013314 |
| 0.74 | -0.0153608 | 0.0037205 | -0.0226528 | -0.0080688 | 0.0000365 | 0.4014311 |
| 0.75 | -0.0152946 | 0.0036552 | -0.0224586 | -0.0081307 | 0.0000286 | 0.4015142 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.01614 Min. :0.003655 Min. :-0.02592
1st Qu.:0.5625 1st Qu.:-0.01603 1st Qu.:0.004042 1st Qu.:-0.02525
Median :0.6250 Median :-0.01588 Median :0.004387 Median :-0.02447
Mean :0.6250 Mean :-0.01582 Mean :0.004363 Mean :-0.02437
3rd Qu.:0.6875 3rd Qu.:-0.01564 3rd Qu.:0.004700 3rd Qu.:-0.02357
Max. :0.7500 Max. :-0.01529 Max. :0.004986 Max. :-0.02246
ci.ub pvalue sigma2
Min. :-0.008131 Min. :2.859e-05 Min. :0.3961
1st Qu.:-0.007721 1st Qu.:1.091e-04 1st Qu.:0.3979
Median :-0.007278 Median :2.963e-04 Median :0.3995
Mean :-0.007269 Mean :4.121e-04 Mean :0.3993
3rd Qu.:-0.006824 3rd Qu.:6.455e-04 3rd Qu.:0.4007
Max. :-0.006369 Max. :1.207e-03 Max. :0.4015
Setting
Code
## SETTING ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$setting <- as.factor(dataN33_prepost$setting)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = setting)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.1725533 | 0.2739199 | -0.7094264 | 0.3643199 | 0.5287334 | 0.4447521 |
| 0.51 | -0.1754365 | 0.2733160 | -0.7111260 | 0.3602530 | 0.5209501 | 0.4455022 |
| 0.52 | -0.1783667 | 0.2726990 | -0.7128468 | 0.3561135 | 0.5130609 | 0.4462607 |
| 0.53 | -0.1813450 | 0.2720684 | -0.7145893 | 0.3518993 | 0.5050648 | 0.4470280 |
| 0.54 | -0.1843727 | 0.2714239 | -0.7163538 | 0.3476084 | 0.4969607 | 0.4478045 |
| 0.55 | -0.1874511 | 0.2707651 | -0.7181408 | 0.3432387 | 0.4887479 | 0.4485904 |
| 0.56 | -0.1905814 | 0.2700914 | -0.7199508 | 0.3387879 | 0.4804255 | 0.4493860 |
| 0.57 | -0.1937651 | 0.2694024 | -0.7217841 | 0.3342539 | 0.4719929 | 0.4501919 |
| 0.58 | -0.1970034 | 0.2686976 | -0.7236411 | 0.3296343 | 0.4634494 | 0.4510082 |
| 0.59 | -0.2002979 | 0.2679766 | -0.7255224 | 0.3249266 | 0.4547947 | 0.4518355 |
| 0.60 | -0.2036500 | 0.2672387 | -0.7274283 | 0.3201282 | 0.4460285 | 0.4526742 |
| 0.61 | -0.2070613 | 0.2664835 | -0.7293593 | 0.3152367 | 0.4371506 | 0.4535245 |
| 0.62 | -0.2105334 | 0.2657102 | -0.7313158 | 0.3102491 | 0.4281612 | 0.4543870 |
| 0.63 | -0.2140679 | 0.2649184 | -0.7332983 | 0.3051626 | 0.4190606 | 0.4552622 |
| 0.64 | -0.2176664 | 0.2641073 | -0.7353072 | 0.2999744 | 0.4098492 | 0.4561504 |
| 0.65 | -0.2213308 | 0.2632763 | -0.7373429 | 0.2946813 | 0.4005279 | 0.4570523 |
| 0.66 | -0.2250629 | 0.2624247 | -0.7394058 | 0.2892801 | 0.3910977 | 0.4579682 |
| 0.67 | -0.2288644 | 0.2615517 | -0.7414964 | 0.2837676 | 0.3815600 | 0.4588988 |
| 0.68 | -0.2327374 | 0.2606566 | -0.7436150 | 0.2781401 | 0.3719166 | 0.4598445 |
| 0.69 | -0.2366839 | 0.2597385 | -0.7457620 | 0.2723943 | 0.3621694 | 0.4608060 |
| 0.70 | -0.2407058 | 0.2587966 | -0.7479378 | 0.2665262 | 0.3523211 | 0.4617837 |
| 0.71 | -0.2448053 | 0.2578299 | -0.7501426 | 0.2605320 | 0.3423746 | 0.4627784 |
| 0.72 | -0.2489846 | 0.2568375 | -0.7523769 | 0.2544077 | 0.3323334 | 0.4637905 |
| 0.73 | -0.2532459 | 0.2558184 | -0.7546408 | 0.2481490 | 0.3222015 | 0.4648207 |
| 0.74 | -0.2575916 | 0.2547715 | -0.7569345 | 0.2417514 | 0.3119834 | 0.4658698 |
| 0.75 | -0.2620240 | 0.2536957 | -0.7592584 | 0.2352105 | 0.3016845 | 0.4669382 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.2620 Min. :0.2537 Min. :-0.7593
1st Qu.:0.5625 1st Qu.:-0.2357 1st Qu.:0.2600 1st Qu.:-0.7452
Median :0.6250 Median :-0.2123 Median :0.2653 Median :-0.7323
Mean :0.6250 Mean :-0.2141 Mean :0.2648 Mean :-0.7330
3rd Qu.:0.6875 3rd Qu.:-0.1914 3rd Qu.:0.2699 3rd Qu.:-0.7204
Max. :0.7500 Max. :-0.1726 Max. :0.2739 Max. :-0.7094
ci.ub pvalue sigma2
Min. :0.2352 Min. :0.3017 Min. :0.4448
1st Qu.:0.2738 1st Qu.:0.3646 1st Qu.:0.4496
Median :0.3077 Median :0.4236 Median :0.4548
Mean :0.3049 Mean :0.4206 Mean :0.4552
3rd Qu.:0.3377 3rd Qu.:0.4783 3rd Qu.:0.4606
Max. :0.3643 Max. :0.5287 Max. :0.4669
Educational level
Code
## EDUCATIONAL LEVEL (dummy-coded) ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$edu <- as.factor(dataN33_prepost$education_dummy)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = edu)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.4825145 | 0.3258198 | -0.1560805 | 1.121110 | 0.1386267 | 0.3837417 |
| 0.51 | 0.4826074 | 0.3257584 | -0.1558673 | 1.121082 | 0.1384763 | 0.3838775 |
| 0.52 | 0.4826918 | 0.3256956 | -0.1556599 | 1.121043 | 0.1383314 | 0.3840102 |
| 0.53 | 0.4827671 | 0.3256313 | -0.1554585 | 1.120993 | 0.1381920 | 0.3841396 |
| 0.54 | 0.4828332 | 0.3255655 | -0.1552635 | 1.120930 | 0.1380585 | 0.3842656 |
| 0.55 | 0.4828896 | 0.3254981 | -0.1550749 | 1.120854 | 0.1379308 | 0.3843880 |
| 0.56 | 0.4829361 | 0.3254290 | -0.1548931 | 1.120765 | 0.1378094 | 0.3845066 |
| 0.57 | 0.4829720 | 0.3253582 | -0.1547184 | 1.120662 | 0.1376945 | 0.3846213 |
| 0.58 | 0.4829971 | 0.3252856 | -0.1545509 | 1.120545 | 0.1375861 | 0.3847318 |
| 0.59 | 0.4830110 | 0.3252111 | -0.1543910 | 1.120413 | 0.1374848 | 0.3848382 |
| 0.60 | 0.4830130 | 0.3251346 | -0.1542391 | 1.120265 | 0.1373906 | 0.3849400 |
| 0.61 | 0.4830027 | 0.3250560 | -0.1540954 | 1.120101 | 0.1373040 | 0.3850371 |
| 0.62 | 0.4829797 | 0.3249753 | -0.1539602 | 1.119920 | 0.1372251 | 0.3851293 |
| 0.63 | 0.4829432 | 0.3248923 | -0.1538340 | 1.119720 | 0.1371544 | 0.3852163 |
| 0.64 | 0.4828928 | 0.3248070 | -0.1537172 | 1.119503 | 0.1370922 | 0.3852978 |
| 0.65 | 0.4828277 | 0.3247191 | -0.1536100 | 1.119266 | 0.1370389 | 0.3853736 |
| 0.66 | 0.4827474 | 0.3246286 | -0.1535130 | 1.119008 | 0.1369948 | 0.3854435 |
| 0.67 | 0.4826510 | 0.3245354 | -0.1534267 | 1.118729 | 0.1369604 | 0.3855069 |
| 0.68 | 0.4825379 | 0.3244392 | -0.1533514 | 1.118427 | 0.1369361 | 0.3855637 |
| 0.69 | 0.4824071 | 0.3243400 | -0.1532877 | 1.118102 | 0.1369225 | 0.3856134 |
| 0.70 | 0.4822578 | 0.3242376 | -0.1532362 | 1.117752 | 0.1369200 | 0.3856556 |
| 0.71 | 0.4820891 | 0.3241318 | -0.1531975 | 1.117376 | 0.1369292 | 0.3856899 |
| 0.72 | 0.4818999 | 0.3240223 | -0.1531721 | 1.116972 | 0.1369506 | 0.3857159 |
| 0.73 | 0.4816891 | 0.3239090 | -0.1531608 | 1.116539 | 0.1369851 | 0.3857329 |
| 0.74 | 0.4814557 | 0.3237916 | -0.1531643 | 1.116076 | 0.1370331 | 0.3857405 |
| 0.75 | 0.4811982 | 0.3236700 | -0.1531832 | 1.115580 | 0.1370956 | 0.3857381 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.4812 Min. :0.3237 Min. :-0.1561
1st Qu.:0.5625 1st Qu.:0.4824 1st Qu.:0.3244 1st Qu.:-0.1548
Median :0.6250 Median :0.4828 Median :0.3249 Median :-0.1539
Mean :0.6250 Mean :0.4826 Mean :0.3249 Mean :-0.1542
3rd Qu.:0.6875 3rd Qu.:0.4829 3rd Qu.:0.3254 3rd Qu.:-0.1533
Max. :0.7500 Max. :0.4830 Max. :0.3258 Max. :-0.1532
ci.ub pvalue sigma2
Min. :1.116 Min. :0.1369 Min. :0.3837
1st Qu.:1.118 1st Qu.:0.1370 1st Qu.:0.3845
Median :1.120 Median :0.1372 Median :0.3852
Mean :1.119 Mean :0.1374 Mean :0.3850
3rd Qu.:1.121 3rd Qu.:0.1378 3rd Qu.:0.3856
Max. :1.121 Max. :0.1386 Max. :0.3857
Post measure
Code
##POST MEASURE (second vs. final draft)
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$post_measure <- as.factor(dataN33_prepost$post_measure)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = post_measure)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.2549189 | 0.3215869 | -0.8852175 | 0.3753798 | 0.4279581 | 0.4054589 |
| 0.51 | -0.2547241 | 0.3216053 | -0.8850588 | 0.3756106 | 0.4283376 | 0.4057681 |
| 0.52 | -0.2545233 | 0.3216219 | -0.8848907 | 0.3758441 | 0.4287257 | 0.4060730 |
| 0.53 | -0.2543161 | 0.3216368 | -0.8847127 | 0.3760804 | 0.4291228 | 0.4063734 |
| 0.54 | -0.2541024 | 0.3216498 | -0.8845244 | 0.3763195 | 0.4295294 | 0.4066692 |
| 0.55 | -0.2538819 | 0.3216608 | -0.8843255 | 0.3765616 | 0.4299457 | 0.4069602 |
| 0.56 | -0.2536543 | 0.3216698 | -0.8841155 | 0.3768068 | 0.4303721 | 0.4072461 |
| 0.57 | -0.2534194 | 0.3216766 | -0.8838940 | 0.3770552 | 0.4308091 | 0.4075267 |
| 0.58 | -0.2531768 | 0.3216812 | -0.8836604 | 0.3773068 | 0.4312570 | 0.4078018 |
| 0.59 | -0.2529263 | 0.3216835 | -0.8834144 | 0.3775619 | 0.4317163 | 0.4080711 |
| 0.60 | -0.2526674 | 0.3216834 | -0.8831553 | 0.3778205 | 0.4321876 | 0.4083344 |
| 0.61 | -0.2523999 | 0.3216807 | -0.8828826 | 0.3780827 | 0.4326713 | 0.4085913 |
| 0.62 | -0.2521235 | 0.3216754 | -0.8825956 | 0.3783487 | 0.4331679 | 0.4088415 |
| 0.63 | -0.2518376 | 0.3216672 | -0.8822938 | 0.3786186 | 0.4336780 | 0.4090847 |
| 0.64 | -0.2515419 | 0.3216562 | -0.8819764 | 0.3788927 | 0.4342023 | 0.4093206 |
| 0.65 | -0.2512359 | 0.3216421 | -0.8816428 | 0.3791709 | 0.4347413 | 0.4095488 |
| 0.66 | -0.2509193 | 0.3216247 | -0.8812921 | 0.3794536 | 0.4352957 | 0.4097688 |
| 0.67 | -0.2505914 | 0.3216039 | -0.8809235 | 0.3797408 | 0.4358664 | 0.4099803 |
| 0.68 | -0.2502517 | 0.3215796 | -0.8805361 | 0.3800327 | 0.4364539 | 0.4101828 |
| 0.69 | -0.2498997 | 0.3215515 | -0.8801291 | 0.3803296 | 0.4370593 | 0.4103757 |
| 0.70 | -0.2495348 | 0.3215194 | -0.8797013 | 0.3806317 | 0.4376833 | 0.4105586 |
| 0.71 | -0.2491563 | 0.3214831 | -0.8792517 | 0.3809390 | 0.4383270 | 0.4107308 |
| 0.72 | -0.2487636 | 0.3214424 | -0.8787791 | 0.3812520 | 0.4389912 | 0.4108918 |
| 0.73 | -0.2483558 | 0.3213970 | -0.8782823 | 0.3815707 | 0.4396772 | 0.4110409 |
| 0.74 | -0.2479322 | 0.3213466 | -0.8777599 | 0.3818955 | 0.4403861 | 0.4111773 |
| 0.75 | -0.2474919 | 0.3212909 | -0.8772104 | 0.3822266 | 0.4411191 | 0.4113003 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.2549 Min. :0.3213 Min. :-0.8852
1st Qu.:0.5625 1st Qu.:-0.2536 1st Qu.:0.3216 1st Qu.:-0.8841
Median :0.6250 Median :-0.2520 Median :0.3216 Median :-0.8824
Mean :0.6250 Mean :-0.2517 Mean :0.3216 Mean :-0.8820
3rd Qu.:0.6875 3rd Qu.:-0.2500 3rd Qu.:0.3217 3rd Qu.:-0.8802
Max. :0.7500 Max. :-0.2475 Max. :0.3217 Max. :-0.8772
ci.ub pvalue sigma2
Min. :0.3754 Min. :0.4280 Min. :0.4055
1st Qu.:0.3769 1st Qu.:0.4305 1st Qu.:0.4073
Median :0.3785 Median :0.4334 Median :0.4090
Mean :0.3786 Mean :0.4338 Mean :0.4088
3rd Qu.:0.3803 3rd Qu.:0.4369 3rd Qu.:0.4103
Max. :0.3822 Max. :0.4411 Max. :0.4113
Amount
Code
##AMOUNT
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$amount <- as.factor(dataN33_prepost$amount)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = amount)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.0773648 | 0.3013705 | -0.6680401 | 0.5133104 | 0.7974026 | 0.4221423 |
| 0.51 | -0.0772197 | 0.3012579 | -0.6676743 | 0.5132348 | 0.7977003 | 0.4223859 |
| 0.52 | -0.0770689 | 0.3011431 | -0.6672985 | 0.5131607 | 0.7980116 | 0.4226248 |
| 0.53 | -0.0769122 | 0.3010261 | -0.6669124 | 0.5130880 | 0.7983368 | 0.4228588 |
| 0.54 | -0.0767494 | 0.3009067 | -0.6665158 | 0.5130169 | 0.7986764 | 0.4230877 |
| 0.55 | -0.0765804 | 0.3007850 | -0.6661082 | 0.5129474 | 0.7990307 | 0.4233113 |
| 0.56 | -0.0764049 | 0.3006609 | -0.6656894 | 0.5128796 | 0.7994003 | 0.4235294 |
| 0.57 | -0.0762228 | 0.3005342 | -0.6652591 | 0.5128134 | 0.7997857 | 0.4237416 |
| 0.58 | -0.0760339 | 0.3004050 | -0.6648169 | 0.5127490 | 0.8001872 | 0.4239478 |
| 0.59 | -0.0758380 | 0.3002731 | -0.6643625 | 0.5126864 | 0.8006056 | 0.4241477 |
| 0.60 | -0.0756348 | 0.3001384 | -0.6638953 | 0.5126257 | 0.8010412 | 0.4243410 |
| 0.61 | -0.0754241 | 0.3000009 | -0.6634152 | 0.5125669 | 0.8014948 | 0.4245273 |
| 0.62 | -0.0752057 | 0.2998605 | -0.6629215 | 0.5125101 | 0.8019670 | 0.4247064 |
| 0.63 | -0.0749793 | 0.2997170 | -0.6624139 | 0.5124553 | 0.8024583 | 0.4248779 |
| 0.64 | -0.0747446 | 0.2995704 | -0.6618918 | 0.5124027 | 0.8029695 | 0.4250414 |
| 0.65 | -0.0745013 | 0.2994205 | -0.6613548 | 0.5123522 | 0.8035013 | 0.4251966 |
| 0.66 | -0.0742492 | 0.2992673 | -0.6608023 | 0.5123040 | 0.8040546 | 0.4253430 |
| 0.67 | -0.0739878 | 0.2991105 | -0.6602337 | 0.5122581 | 0.8046301 | 0.4254801 |
| 0.68 | -0.0737169 | 0.2989501 | -0.6596483 | 0.5122146 | 0.8052287 | 0.4256075 |
| 0.69 | -0.0734360 | 0.2987859 | -0.6590456 | 0.5121736 | 0.8058515 | 0.4257246 |
| 0.70 | -0.0731448 | 0.2986177 | -0.6584248 | 0.5121352 | 0.8064992 | 0.4258309 |
| 0.71 | -0.0728429 | 0.2984454 | -0.6577852 | 0.5120994 | 0.8071732 | 0.4259257 |
| 0.72 | -0.0725297 | 0.2982688 | -0.6571259 | 0.5120664 | 0.8078744 | 0.4260085 |
| 0.73 | -0.0722049 | 0.2980877 | -0.6564461 | 0.5120363 | 0.8086043 | 0.4260785 |
| 0.74 | -0.0718678 | 0.2979019 | -0.6557448 | 0.5120092 | 0.8093639 | 0.4261349 |
| 0.75 | -0.0715180 | 0.2977111 | -0.6550211 | 0.5119851 | 0.8101549 | 0.4261769 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.07736 Min. :0.2977 Min. :-0.6680
1st Qu.:0.5625 1st Qu.:-0.07636 1st Qu.:0.2988 1st Qu.:-0.6656
Median :0.6250 Median :-0.07509 Median :0.2998 Median :-0.6627
Mean :0.6250 Mean :-0.07486 Mean :0.2997 Mean :-0.6623
3rd Qu.:0.6875 3rd Qu.:-0.07351 3rd Qu.:0.3006 3rd Qu.:-0.6592
Max. :0.7500 Max. :-0.07152 Max. :0.3014 Max. :-0.6550
ci.ub pvalue sigma2
Min. :0.5120 Min. :0.7974 Min. :0.4221
1st Qu.:0.5122 1st Qu.:0.7995 1st Qu.:0.4236
Median :0.5125 Median :0.8022 Median :0.4248
Mean :0.5125 Mean :0.8028 Mean :0.4246
3rd Qu.:0.5129 3rd Qu.:0.8057 3rd Qu.:0.4257
Max. :0.5133 Max. :0.8102 Max. :0.4262
###Coding
Code
###CODING
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$coding <- as.factor(dataN33_prepost$coding)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = coding)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.2688236 | 0.2632461 | -0.7847764 | 0.2471291 | 0.3071655 | 0.3916269 |
| 0.51 | -0.2686162 | 0.2632893 | -0.7846537 | 0.2474214 | 0.3076185 | 0.3919788 |
| 0.52 | -0.2684039 | 0.2633311 | -0.7845233 | 0.2477156 | 0.3080778 | 0.3923265 |
| 0.53 | -0.2681865 | 0.2633714 | -0.7843849 | 0.2480119 | 0.3085437 | 0.3926697 |
| 0.54 | -0.2679639 | 0.2634101 | -0.7842382 | 0.2483103 | 0.3090164 | 0.3930084 |
| 0.55 | -0.2677359 | 0.2634471 | -0.7840828 | 0.2486110 | 0.3094963 | 0.3933423 |
| 0.56 | -0.2675022 | 0.2634824 | -0.7839183 | 0.2489139 | 0.3099836 | 0.3936712 |
| 0.57 | -0.2672627 | 0.2635160 | -0.7837445 | 0.2492192 | 0.3104787 | 0.3939948 |
| 0.58 | -0.2670170 | 0.2635477 | -0.7835609 | 0.2495269 | 0.3109818 | 0.3943130 |
| 0.59 | -0.2667650 | 0.2635774 | -0.7833671 | 0.2498372 | 0.3114933 | 0.3946255 |
| 0.60 | -0.2665063 | 0.2636051 | -0.7831627 | 0.2501501 | 0.3120136 | 0.3949319 |
| 0.61 | -0.2662407 | 0.2636306 | -0.7829472 | 0.2504659 | 0.3125430 | 0.3952321 |
| 0.62 | -0.2659678 | 0.2636539 | -0.7827200 | 0.2507844 | 0.3130820 | 0.3955257 |
| 0.63 | -0.2656874 | 0.2636749 | -0.7824808 | 0.2511059 | 0.3136309 | 0.3958125 |
| 0.64 | -0.2653992 | 0.2636935 | -0.7822288 | 0.2514305 | 0.3141903 | 0.3960919 |
| 0.65 | -0.2651026 | 0.2637094 | -0.7819636 | 0.2517584 | 0.3147606 | 0.3963638 |
| 0.66 | -0.2647975 | 0.2637227 | -0.7816845 | 0.2520895 | 0.3153423 | 0.3966277 |
| 0.67 | -0.2644833 | 0.2637332 | -0.7813908 | 0.2524242 | 0.3159359 | 0.3968831 |
| 0.68 | -0.2641597 | 0.2637406 | -0.7810818 | 0.2527624 | 0.3165421 | 0.3971296 |
| 0.69 | -0.2638262 | 0.2637449 | -0.7807567 | 0.2531044 | 0.3171615 | 0.3973668 |
| 0.70 | -0.2634822 | 0.2637459 | -0.7804147 | 0.2534503 | 0.3177947 | 0.3975941 |
| 0.71 | -0.2631273 | 0.2637434 | -0.7800549 | 0.2538003 | 0.3184424 | 0.3978109 |
| 0.72 | -0.2627608 | 0.2637372 | -0.7796763 | 0.2541546 | 0.3191054 | 0.3980167 |
| 0.73 | -0.2623823 | 0.2637271 | -0.7792778 | 0.2545133 | 0.3197845 | 0.3982107 |
| 0.74 | -0.2619910 | 0.2637128 | -0.7788585 | 0.2548766 | 0.3204806 | 0.3983923 |
| 0.75 | -0.2615861 | 0.2636941 | -0.7784170 | 0.2552448 | 0.3211946 | 0.3985606 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.2688 Min. :0.2632 Min. :-0.7848
1st Qu.:0.5625 1st Qu.:-0.2674 1st Qu.:0.2635 1st Qu.:-0.7839
Median :0.6250 Median :-0.2658 Median :0.2637 Median :-0.7826
Mean :0.6250 Mean :-0.2656 Mean :0.2636 Mean :-0.7822
3rd Qu.:0.6875 3rd Qu.:-0.2639 3rd Qu.:0.2637 3rd Qu.:-0.7808
Max. :0.7500 Max. :-0.2616 Max. :0.2637 Max. :-0.7784
ci.ub pvalue sigma2
Min. :0.2471 Min. :0.3072 Min. :0.3916
1st Qu.:0.2490 1st Qu.:0.3101 1st Qu.:0.3938
Median :0.2509 Median :0.3134 Median :0.3957
Mean :0.2510 Mean :0.3136 Mean :0.3955
3rd Qu.:0.2530 3rd Qu.:0.3170 3rd Qu.:0.3973
Max. :0.2552 Max. :0.3212 Max. :0.3986
###Level of outcome
Code
##LEVEL OF OUTCOME
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$order_outcome <- as.factor(dataN33_prepost$order_outcome)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = order_outcome)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.0402285 | 0.2089137 | -0.4496918 | 0.3692347 | 0.8473031 | 0.4337076 |
| 0.51 | -0.0402219 | 0.2089335 | -0.4497241 | 0.3692803 | 0.8473423 | 0.4339782 |
| 0.52 | -0.0402136 | 0.2089522 | -0.4497523 | 0.3693251 | 0.8473869 | 0.4342436 |
| 0.53 | -0.0402036 | 0.2089695 | -0.4497762 | 0.3693691 | 0.8474370 | 0.4345037 |
| 0.54 | -0.0401918 | 0.2089855 | -0.4497958 | 0.3694122 | 0.8474928 | 0.4347581 |
| 0.55 | -0.0401781 | 0.2090000 | -0.4498107 | 0.3694544 | 0.8475544 | 0.4350067 |
| 0.56 | -0.0401626 | 0.2090132 | -0.4498209 | 0.3694956 | 0.8476219 | 0.4352492 |
| 0.57 | -0.0401452 | 0.2090247 | -0.4498262 | 0.3695357 | 0.8476954 | 0.4354852 |
| 0.58 | -0.0401259 | 0.2090347 | -0.4498265 | 0.3695747 | 0.8477751 | 0.4357147 |
| 0.59 | -0.0401045 | 0.2090431 | -0.4498214 | 0.3696124 | 0.8478611 | 0.4359371 |
| 0.60 | -0.0400811 | 0.2090497 | -0.4498109 | 0.3696487 | 0.8479535 | 0.4361522 |
| 0.61 | -0.0400556 | 0.2090544 | -0.4497948 | 0.3696835 | 0.8480525 | 0.4363597 |
| 0.62 | -0.0400280 | 0.2090573 | -0.4497728 | 0.3697168 | 0.8481581 | 0.4365593 |
| 0.63 | -0.0399982 | 0.2090582 | -0.4497447 | 0.3697483 | 0.8482705 | 0.4367504 |
| 0.64 | -0.0399661 | 0.2090569 | -0.4497102 | 0.3697780 | 0.8483898 | 0.4369327 |
| 0.65 | -0.0399318 | 0.2090535 | -0.4496691 | 0.3698056 | 0.8485161 | 0.4371059 |
| 0.66 | -0.0398951 | 0.2090478 | -0.4496212 | 0.3698311 | 0.8486495 | 0.4372693 |
| 0.67 | -0.0398560 | 0.2090396 | -0.4495662 | 0.3698542 | 0.8487900 | 0.4374225 |
| 0.68 | -0.0398146 | 0.2090289 | -0.4495038 | 0.3698746 | 0.8489378 | 0.4375649 |
| 0.69 | -0.0397707 | 0.2090156 | -0.4494336 | 0.3698923 | 0.8490928 | 0.4376961 |
| 0.70 | -0.0397243 | 0.2089993 | -0.4493555 | 0.3699069 | 0.8492550 | 0.4378153 |
| 0.71 | -0.0396754 | 0.2089801 | -0.4492689 | 0.3699180 | 0.8494246 | 0.4379218 |
| 0.72 | -0.0396241 | 0.2089577 | -0.4491737 | 0.3699255 | 0.8496013 | 0.4380151 |
| 0.73 | -0.0395702 | 0.2089320 | -0.4490693 | 0.3699290 | 0.8497850 | 0.4380942 |
| 0.74 | -0.0395138 | 0.2089027 | -0.4489555 | 0.3699279 | 0.8499757 | 0.4381583 |
| 0.75 | -0.0394550 | 0.2088696 | -0.4488319 | 0.3699220 | 0.8501731 | 0.4382066 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.04023 Min. :0.2089 Min. :-0.4498
1st Qu.:0.5625 1st Qu.:-0.04016 1st Qu.:0.2090 1st Qu.:-0.4498
Median :0.6250 Median :-0.04001 Median :0.2090 Median :-0.4497
Mean :0.6250 Mean :-0.03995 Mean :0.2090 Mean :-0.4496
3rd Qu.:0.6875 3rd Qu.:-0.03978 3rd Qu.:0.2090 3rd Qu.:-0.4495
Max. :0.7500 Max. :-0.03945 Max. :0.2091 Max. :-0.4488
ci.ub pvalue sigma2
Min. :0.3692 Min. :0.8473 Min. :0.4337
1st Qu.:0.3695 1st Qu.:0.8476 1st Qu.:0.4353
Median :0.3697 Median :0.8482 Median :0.4367
Mean :0.3697 Mean :0.8484 Mean :0.4364
3rd Qu.:0.3699 3rd Qu.:0.8491 3rd Qu.:0.4377
Max. :0.3699 Max. :0.8502 Max. :0.4382
###Experiment
Code
##EXPERIMENT
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$experiment <- as.factor(dataN33_prepost$experiment)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = experiment)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.6957096 | 0.4044346 | -1.488387 | 0.0969677 | 0.0853956 | 0.3639792 |
| 0.51 | -0.6956866 | 0.4044468 | -1.488388 | 0.0970145 | 0.0854153 | 0.3643096 |
| 0.52 | -0.6956627 | 0.4044563 | -1.488382 | 0.0970570 | 0.0854333 | 0.3646351 |
| 0.53 | -0.6956381 | 0.4044630 | -1.488371 | 0.0970947 | 0.0854496 | 0.3649554 |
| 0.54 | -0.6956126 | 0.4044667 | -1.488353 | 0.0971276 | 0.0854640 | 0.3652703 |
| 0.55 | -0.6955863 | 0.4044675 | -1.488328 | 0.0971555 | 0.0854764 | 0.3655796 |
| 0.56 | -0.6955591 | 0.4044652 | -1.488296 | 0.0971780 | 0.0854868 | 0.3658829 |
| 0.57 | -0.6955310 | 0.4044595 | -1.488257 | 0.0971951 | 0.0854951 | 0.3661801 |
| 0.58 | -0.6955020 | 0.4044505 | -1.488210 | 0.0972063 | 0.0855011 | 0.3664709 |
| 0.59 | -0.6954720 | 0.4044378 | -1.488156 | 0.0972116 | 0.0855049 | 0.3667549 |
| 0.60 | -0.6954410 | 0.4044215 | -1.488093 | 0.0972105 | 0.0855061 | 0.3670319 |
| 0.61 | -0.6954090 | 0.4044012 | -1.488021 | 0.0972027 | 0.0855048 | 0.3673015 |
| 0.62 | -0.6953760 | 0.4043769 | -1.487940 | 0.0971880 | 0.0855009 | 0.3675634 |
| 0.63 | -0.6953420 | 0.4043482 | -1.487850 | 0.0971660 | 0.0854940 | 0.3678171 |
| 0.64 | -0.6953069 | 0.4043151 | -1.487750 | 0.0971362 | 0.0854842 | 0.3680623 |
| 0.65 | -0.6952707 | 0.4042773 | -1.487640 | 0.0970982 | 0.0854712 | 0.3682985 |
| 0.66 | -0.6952335 | 0.4042345 | -1.487519 | 0.0970516 | 0.0854549 | 0.3685253 |
| 0.67 | -0.6951952 | 0.4041865 | -1.487386 | 0.0969958 | 0.0854350 | 0.3687421 |
| 0.68 | -0.6951559 | 0.4041330 | -1.487242 | 0.0969303 | 0.0854113 | 0.3689484 |
| 0.69 | -0.6951156 | 0.4040738 | -1.487086 | 0.0968545 | 0.0853836 | 0.3691437 |
| 0.70 | -0.6950743 | 0.4040084 | -1.486916 | 0.0967677 | 0.0853516 | 0.3693272 |
| 0.71 | -0.6950321 | 0.4039366 | -1.486733 | 0.0966690 | 0.0853149 | 0.3694984 |
| 0.72 | -0.6949892 | 0.4038579 | -1.486536 | 0.0965576 | 0.0852734 | 0.3696564 |
| 0.73 | -0.6949458 | 0.4037721 | -1.486325 | 0.0964330 | 0.0852266 | 0.3698009 |
| 0.74 | -0.6949019 | 0.4036784 | -1.486097 | 0.0962933 | 0.0851738 | 0.3699300 |
| 0.75 | -0.6948578 | 0.4035765 | -1.485853 | 0.0961377 | 0.0851149 | 0.3700434 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.6957 Min. :0.4036 Min. :-1.488
1st Qu.:0.5625 1st Qu.:-0.6956 1st Qu.:0.4041 1st Qu.:-1.488
Median :0.6250 Median :-0.6954 Median :0.4044 Median :-1.488
Mean :0.6250 Mean :-0.6953 Mean :0.4042 Mean :-1.488
3rd Qu.:0.6875 3rd Qu.:-0.6951 3rd Qu.:0.4044 3rd Qu.:-1.487
Max. :0.7500 Max. :-0.6949 Max. :0.4045 Max. :-1.486
ci.ub pvalue sigma2
Min. :0.09614 Min. :0.08511 Min. :0.3640
1st Qu.:0.09687 1st Qu.:0.08539 1st Qu.:0.3660
Median :0.09708 Median :0.08545 Median :0.3677
Mean :0.09696 Mean :0.08541 Mean :0.3675
3rd Qu.:0.09718 3rd Qu.:0.08549 3rd Qu.:0.3691
Max. :0.09721 Max. :0.08551 Max. :0.3700
###System
Code
##SYSTEM
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$system <- as.factor(dataN33_prepost$system_type)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = system)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.1069393 | 0.1124696 | -0.3273757 | 0.1134971 | 0.3416914 | 0.4073082 |
| 0.51 | -0.1069519 | 0.1124791 | -0.3274068 | 0.1135031 | 0.3416754 | 0.4076081 |
| 0.52 | -0.1069649 | 0.1124878 | -0.3274370 | 0.1135071 | 0.3416539 | 0.4079024 |
| 0.53 | -0.1069785 | 0.1124957 | -0.3274661 | 0.1135091 | 0.3416268 | 0.4081909 |
| 0.54 | -0.1069926 | 0.1125029 | -0.3274941 | 0.1135090 | 0.3415937 | 0.4084734 |
| 0.55 | -0.1070072 | 0.1125091 | -0.3275210 | 0.1135066 | 0.3415546 | 0.4087497 |
| 0.56 | -0.1070224 | 0.1125145 | -0.3275467 | 0.1135020 | 0.3415092 | 0.4090194 |
| 0.57 | -0.1070381 | 0.1125189 | -0.3275712 | 0.1134949 | 0.3414572 | 0.4092822 |
| 0.58 | -0.1070545 | 0.1125224 | -0.3275944 | 0.1134854 | 0.3413983 | 0.4095380 |
| 0.59 | -0.1070715 | 0.1125249 | -0.3276162 | 0.1134732 | 0.3413324 | 0.4097863 |
| 0.60 | -0.1070891 | 0.1125263 | -0.3276366 | 0.1134584 | 0.3412590 | 0.4100268 |
| 0.61 | -0.1071074 | 0.1125266 | -0.3276555 | 0.1134407 | 0.3411779 | 0.4102592 |
| 0.62 | -0.1071263 | 0.1125258 | -0.3276728 | 0.1134201 | 0.3410888 | 0.4104830 |
| 0.63 | -0.1071460 | 0.1125237 | -0.3276885 | 0.1133964 | 0.3409913 | 0.4106980 |
| 0.64 | -0.1071665 | 0.1125204 | -0.3277024 | 0.1133695 | 0.3408851 | 0.4109036 |
| 0.65 | -0.1071877 | 0.1125158 | -0.3277145 | 0.1133392 | 0.3407696 | 0.4110993 |
| 0.66 | -0.1072097 | 0.1125098 | -0.3277248 | 0.1133054 | 0.3406447 | 0.4112848 |
| 0.67 | -0.1072325 | 0.1125023 | -0.3277329 | 0.1132679 | 0.3405096 | 0.4114594 |
| 0.68 | -0.1072563 | 0.1124933 | -0.3277390 | 0.1132265 | 0.3403641 | 0.4116227 |
| 0.69 | -0.1072809 | 0.1124826 | -0.3277428 | 0.1131810 | 0.3402075 | 0.4117739 |
| 0.70 | -0.1073065 | 0.1124703 | -0.3277442 | 0.1131312 | 0.3400393 | 0.4119125 |
| 0.71 | -0.1073331 | 0.1124561 | -0.3277431 | 0.1130769 | 0.3398589 | 0.4120378 |
| 0.72 | -0.1073607 | 0.1124401 | -0.3277393 | 0.1130178 | 0.3396657 | 0.4121490 |
| 0.73 | -0.1073895 | 0.1124221 | -0.3277327 | 0.1129537 | 0.3394589 | 0.4122452 |
| 0.74 | -0.1074194 | 0.1124019 | -0.3277231 | 0.1128843 | 0.3392377 | 0.4123256 |
| 0.75 | -0.1074505 | 0.1123795 | -0.3277103 | 0.1128092 | 0.3390014 | 0.4123893 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.1075 Min. :0.1124 Min. :-0.3277
1st Qu.:0.5625 1st Qu.:-0.1073 1st Qu.:0.1125 1st Qu.:-0.3277
Median :0.6250 Median :-0.1071 Median :0.1125 Median :-0.3277
Mean :0.6250 Mean :-0.1072 Mean :0.1125 Mean :-0.3276
3rd Qu.:0.6875 3rd Qu.:-0.1070 3rd Qu.:0.1125 3rd Qu.:-0.3276
Max. :0.7500 Max. :-0.1069 Max. :0.1125 Max. :-0.3274
ci.ub pvalue sigma2
Min. :0.1128 Min. :0.3390 Min. :0.4073
1st Qu.:0.1132 1st Qu.:0.3402 1st Qu.:0.4091
Median :0.1134 Median :0.3410 Median :0.4106
Mean :0.1133 Mean :0.3408 Mean :0.4103
3rd Qu.:0.1135 3rd Qu.:0.3415 3rd Qu.:0.4117
Max. :0.1135 Max. :0.3417 Max. :0.4124
###Teacher effects
Code
##teacher_effects
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$teacher <- as.factor(dataN33_prepost$teacher_effects)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = teacher)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.1881239 | 0.2753030 | -0.7277079 | 0.3514601 | 0.4943958 | 0.4219364 |
| 0.51 | -0.1882207 | 0.2752857 | -0.7277707 | 0.3513292 | 0.4941464 | 0.4222099 |
| 0.52 | -0.1883190 | 0.2752665 | -0.7278315 | 0.3511935 | 0.4938909 | 0.4224781 |
| 0.53 | -0.1884189 | 0.2752456 | -0.7278902 | 0.3510525 | 0.4936291 | 0.4227405 |
| 0.54 | -0.1885202 | 0.2752227 | -0.7279468 | 0.3509063 | 0.4933607 | 0.4229970 |
| 0.55 | -0.1886232 | 0.2751978 | -0.7280010 | 0.3507545 | 0.4930855 | 0.4232474 |
| 0.56 | -0.1887279 | 0.2751708 | -0.7280528 | 0.3505970 | 0.4928032 | 0.4234913 |
| 0.57 | -0.1888343 | 0.2751417 | -0.7281021 | 0.3504335 | 0.4925136 | 0.4237285 |
| 0.58 | -0.1889425 | 0.2751103 | -0.7281487 | 0.3502638 | 0.4922163 | 0.4239587 |
| 0.59 | -0.1890525 | 0.2750765 | -0.7281926 | 0.3500876 | 0.4919111 | 0.4241816 |
| 0.60 | -0.1891645 | 0.2750403 | -0.7282336 | 0.3499047 | 0.4915977 | 0.4243968 |
| 0.61 | -0.1892784 | 0.2750016 | -0.7282716 | 0.3497147 | 0.4912757 | 0.4246040 |
| 0.62 | -0.1893944 | 0.2749601 | -0.7283063 | 0.3495174 | 0.4909447 | 0.4248028 |
| 0.63 | -0.1895126 | 0.2749158 | -0.7283377 | 0.3493124 | 0.4906044 | 0.4249929 |
| 0.64 | -0.1896330 | 0.2748686 | -0.7283655 | 0.3490994 | 0.4902543 | 0.4251737 |
| 0.65 | -0.1897558 | 0.2748182 | -0.7283896 | 0.3488781 | 0.4898940 | 0.4253449 |
| 0.66 | -0.1898809 | 0.2747646 | -0.7284097 | 0.3486479 | 0.4895231 | 0.4255060 |
| 0.67 | -0.1900086 | 0.2747077 | -0.7284257 | 0.3484085 | 0.4891410 | 0.4256564 |
| 0.68 | -0.1901389 | 0.2746471 | -0.7284373 | 0.3481595 | 0.4887473 | 0.4257955 |
| 0.69 | -0.1902720 | 0.2745827 | -0.7284443 | 0.3479003 | 0.4883412 | 0.4259229 |
| 0.70 | -0.1904079 | 0.2745144 | -0.7284463 | 0.3476305 | 0.4879223 | 0.4260378 |
| 0.71 | -0.1905469 | 0.2744419 | -0.7284431 | 0.3473494 | 0.4874898 | 0.4261395 |
| 0.72 | -0.1906890 | 0.2743650 | -0.7284345 | 0.3470565 | 0.4870431 | 0.4262274 |
| 0.73 | -0.1908344 | 0.2742834 | -0.7284199 | 0.3467510 | 0.4865813 | 0.4263006 |
| 0.74 | -0.1909834 | 0.2741968 | -0.7283992 | 0.3464324 | 0.4861035 | 0.4263582 |
| 0.75 | -0.1911360 | 0.2741049 | -0.7283718 | 0.3460997 | 0.4856090 | 0.4263993 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.1911 Min. :0.2741 Min. :-0.7284
1st Qu.:0.5625 1st Qu.:-0.1902 1st Qu.:0.2746 1st Qu.:-0.7284
Median :0.6250 Median :-0.1895 Median :0.2749 Median :-0.7283
Mean :0.6250 Mean :-0.1895 Mean :0.2749 Mean :-0.7282
3rd Qu.:0.6875 3rd Qu.:-0.1888 3rd Qu.:0.2752 3rd Qu.:-0.7281
Max. :0.7500 Max. :-0.1881 Max. :0.2753 Max. :-0.7277
ci.ub pvalue sigma2
Min. :0.3461 Min. :0.4856 Min. :0.4219
1st Qu.:0.3480 1st Qu.:0.4884 1st Qu.:0.4236
Median :0.3494 Median :0.4908 Median :0.4249
Mean :0.3492 Mean :0.4905 Mean :0.4246
3rd Qu.:0.3506 3rd Qu.:0.4927 3rd Qu.:0.4259
Max. :0.3515 Max. :0.4944 Max. :0.4264
###Validated tool
Code
##validated_tool
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$validated_tool <- as.factor(dataN33_prepost$validated_tool)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = validated_tool)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 1.191959 | 0.5804574 | 0.0542833 | 2.329634 | 0.0400258 | 0.3120282 |
| 0.51 | 1.191951 | 0.5807842 | 0.0536350 | 2.330267 | 0.0401392 | 0.3124636 |
| 0.52 | 1.191939 | 0.5811059 | 0.0529922 | 2.330885 | 0.0402518 | 0.3128936 |
| 0.53 | 1.191922 | 0.5814222 | 0.0523550 | 2.331488 | 0.0403634 | 0.3133181 |
| 0.54 | 1.191899 | 0.5817331 | 0.0517235 | 2.332075 | 0.0404742 | 0.3137367 |
| 0.55 | 1.191872 | 0.5820382 | 0.0510982 | 2.332646 | 0.0405839 | 0.3141492 |
| 0.56 | 1.191839 | 0.5823374 | 0.0504790 | 2.333200 | 0.0406927 | 0.3145554 |
| 0.57 | 1.191801 | 0.5826304 | 0.0498664 | 2.333736 | 0.0408004 | 0.3149551 |
| 0.58 | 1.191757 | 0.5829170 | 0.0492605 | 2.334253 | 0.0409071 | 0.3153478 |
| 0.59 | 1.191707 | 0.5831972 | 0.0486610 | 2.334752 | 0.0410126 | 0.3157338 |
| 0.60 | 1.191650 | 0.5834702 | 0.0480695 | 2.335231 | 0.0411169 | 0.3161120 |
| 0.61 | 1.191587 | 0.5837358 | 0.0474857 | 2.335688 | 0.0412198 | 0.3164823 |
| 0.62 | 1.191517 | 0.5839938 | 0.0469099 | 2.336124 | 0.0413214 | 0.3168444 |
| 0.63 | 1.191440 | 0.5842440 | 0.0463425 | 2.336537 | 0.0414217 | 0.3171979 |
| 0.64 | 1.191355 | 0.5844858 | 0.0457838 | 2.336926 | 0.0415204 | 0.3175425 |
| 0.65 | 1.191262 | 0.5847190 | 0.0452343 | 2.337290 | 0.0416175 | 0.3178776 |
| 0.66 | 1.191161 | 0.5849422 | 0.0446958 | 2.337627 | 0.0417128 | 0.3182020 |
| 0.67 | 1.191052 | 0.5851575 | 0.0441646 | 2.337940 | 0.0418067 | 0.3185178 |
| 0.68 | 1.190934 | 0.5853620 | 0.0436455 | 2.338222 | 0.0418986 | 0.3188217 |
| 0.69 | 1.190806 | 0.5855560 | 0.0431376 | 2.338475 | 0.0419886 | 0.3191141 |
| 0.70 | 1.190668 | 0.5857388 | 0.0426415 | 2.338696 | 0.0420764 | 0.3193945 |
| 0.71 | 1.190520 | 0.5859100 | 0.0421579 | 2.338883 | 0.0421621 | 0.3196620 |
| 0.72 | 1.190361 | 0.5860688 | 0.0416876 | 2.339035 | 0.0422454 | 0.3199160 |
| 0.73 | 1.190191 | 0.5862146 | 0.0412313 | 2.339150 | 0.0423262 | 0.3201556 |
| 0.74 | 1.190008 | 0.5863466 | 0.0407900 | 2.339226 | 0.0424043 | 0.3203801 |
| 0.75 | 1.189813 | 0.5864640 | 0.0403645 | 2.339261 | 0.0424796 | 0.3205886 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :1.190 Min. :0.5805 Min. :0.04036
1st Qu.:0.5625 1st Qu.:1.191 1st Qu.:0.5824 1st Qu.:0.04326
Median :0.6250 Median :1.191 Median :0.5841 Median :0.04663
Mean :0.6250 Mean :1.191 Mean :0.5839 Mean :0.04687
3rd Qu.:0.6875 3rd Qu.:1.192 3rd Qu.:0.5855 3rd Qu.:0.05033
Max. :0.7500 Max. :1.192 Max. :0.5865 Max. :0.05428
ci.ub pvalue sigma2
Min. :2.330 Min. :0.04003 Min. :0.3120
1st Qu.:2.333 1st Qu.:0.04072 1st Qu.:0.3147
Median :2.336 Median :0.04137 Median :0.3170
Mean :2.336 Mean :0.04133 Mean :0.3168
3rd Qu.:2.338 3rd Qu.:0.04197 3rd Qu.:0.3190
Max. :2.339 Max. :0.04248 Max. :0.3206
###Reliability of measurement
Code
##reliability_measurement
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$reliability <- as.factor(dataN33_prepost$reliability_measurement)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = reliability)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.0812447 | 0.3057183 | -0.5179522 | 0.6804416 | 0.7904316 | 0.4349500 |
| 0.51 | 0.0816673 | 0.3057335 | -0.5175593 | 0.6808939 | 0.7893774 | 0.4352120 |
| 0.52 | 0.0820959 | 0.3057468 | -0.5171567 | 0.6813486 | 0.7883071 | 0.4354685 |
| 0.53 | 0.0825310 | 0.3057581 | -0.5167439 | 0.6818058 | 0.7872199 | 0.4357192 |
| 0.54 | 0.0829727 | 0.3057674 | -0.5163203 | 0.6822658 | 0.7861149 | 0.4359641 |
| 0.55 | 0.0834216 | 0.3057746 | -0.5158855 | 0.6827288 | 0.7849910 | 0.4362027 |
| 0.56 | 0.0838781 | 0.3057795 | -0.5154388 | 0.6831949 | 0.7838472 | 0.4364349 |
| 0.57 | 0.0843424 | 0.3057822 | -0.5149796 | 0.6836645 | 0.7826823 | 0.4366604 |
| 0.58 | 0.0848153 | 0.3057824 | -0.5145073 | 0.6841378 | 0.7814950 | 0.4368788 |
| 0.59 | 0.0852970 | 0.3057802 | -0.5140211 | 0.6846151 | 0.7802841 | 0.4370898 |
| 0.60 | 0.0857883 | 0.3057753 | -0.5135202 | 0.6850968 | 0.7790480 | 0.4372932 |
| 0.61 | 0.0862896 | 0.3057677 | -0.5130040 | 0.6855832 | 0.7777851 | 0.4374885 |
| 0.62 | 0.0868017 | 0.3057571 | -0.5124713 | 0.6860747 | 0.7764939 | 0.4376753 |
| 0.63 | 0.0873252 | 0.3057436 | -0.5119214 | 0.6865717 | 0.7751725 | 0.4378533 |
| 0.64 | 0.0878608 | 0.3057270 | -0.5113530 | 0.6870747 | 0.7738189 | 0.4380220 |
| 0.65 | 0.0884095 | 0.3057070 | -0.5107652 | 0.6875842 | 0.7724308 | 0.4381810 |
| 0.66 | 0.0889720 | 0.3056835 | -0.5101567 | 0.6881007 | 0.7710060 | 0.4383297 |
| 0.67 | 0.0895494 | 0.3056564 | -0.5095261 | 0.6886250 | 0.7695419 | 0.4384676 |
| 0.68 | 0.0901428 | 0.3056254 | -0.5088720 | 0.6891577 | 0.7680356 | 0.4385941 |
| 0.69 | 0.0907534 | 0.3055904 | -0.5081928 | 0.6896996 | 0.7664840 | 0.4387086 |
| 0.70 | 0.0913823 | 0.3055511 | -0.5074868 | 0.6902514 | 0.7648838 | 0.4388105 |
| 0.71 | 0.0920312 | 0.3055072 | -0.5067520 | 0.6908143 | 0.7632311 | 0.4388989 |
| 0.72 | 0.0927015 | 0.3054586 | -0.5059864 | 0.6913893 | 0.7615219 | 0.4389733 |
| 0.73 | 0.0933950 | 0.3054049 | -0.5051876 | 0.6919776 | 0.7597515 | 0.4390326 |
| 0.74 | 0.0941137 | 0.3053458 | -0.5043530 | 0.6925804 | 0.7579149 | 0.4390759 |
| 0.75 | 0.0948598 | 0.3052809 | -0.5034799 | 0.6931995 | 0.7560063 | 0.4391024 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.08124 Min. :0.3053 Min. :-0.5180
1st Qu.:0.5625 1st Qu.:0.08399 1st Qu.:0.3056 1st Qu.:-0.5153
Median :0.6250 Median :0.08706 Median :0.3057 Median :-0.5122
Mean :0.6250 Mean :0.08741 Mean :0.3057 Mean :-0.5117
3rd Qu.:0.6875 3rd Qu.:0.09060 3rd Qu.:0.3058 3rd Qu.:-0.5084
Max. :0.7500 Max. :0.09486 Max. :0.3058 Max. :-0.5035
ci.ub pvalue sigma2
Min. :0.6804 Min. :0.7560 Min. :0.4350
1st Qu.:0.6833 1st Qu.:0.7669 1st Qu.:0.4365
Median :0.6863 Median :0.7758 Median :0.4378
Mean :0.6865 Mean :0.7749 Mean :0.4375
3rd Qu.:0.6896 3rd Qu.:0.7836 3rd Qu.:0.4387
Max. :0.6932 Max. :0.7904 Max. :0.4391
Code
#not specified
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$reliable_not <- ifelse(dataN33_prepost$reliability_measurement == 2, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = reliable_not)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue |
|---|---|---|---|---|---|
| 0.50 | 0.1223551 | 0.4390444 | -0.7381562 | 0.9828664 | 0.7804866 |
| 0.51 | 0.1229361 | 0.4390222 | -0.7375317 | 0.9834039 | 0.7794602 |
| 0.52 | 0.1235245 | 0.4389974 | -0.7368946 | 0.9839437 | 0.7784200 |
| 0.53 | 0.1241207 | 0.4389699 | -0.7362445 | 0.9844859 | 0.7773650 |
| 0.54 | 0.1247251 | 0.4389396 | -0.7355806 | 0.9850309 | 0.7762946 |
| 0.55 | 0.1253382 | 0.4389063 | -0.7349023 | 0.9855787 | 0.7752079 |
| 0.56 | 0.1259604 | 0.4388699 | -0.7342088 | 0.9861296 | 0.7741040 |
| 0.57 | 0.1265923 | 0.4388304 | -0.7334994 | 0.9866840 | 0.7729820 |
| 0.58 | 0.1272344 | 0.4387875 | -0.7327733 | 0.9872420 | 0.7718406 |
| 0.59 | 0.1278873 | 0.4387411 | -0.7320295 | 0.9878041 | 0.7706790 |
| 0.60 | 0.1285517 | 0.4386911 | -0.7312671 | 0.9883705 | 0.7694957 |
| 0.61 | 0.1292282 | 0.4386373 | -0.7304852 | 0.9889416 | 0.7682896 |
| 0.62 | 0.1299177 | 0.4385796 | -0.7296825 | 0.9895178 | 0.7670592 |
| 0.63 | 0.1306209 | 0.4385176 | -0.7288578 | 0.9900997 | 0.7658029 |
| 0.64 | 0.1313388 | 0.4384513 | -0.7280100 | 0.9906877 | 0.7645192 |
| 0.65 | 0.1320723 | 0.4383805 | -0.7271376 | 0.9912823 | 0.7632061 |
| 0.66 | 0.1328225 | 0.4383048 | -0.7262391 | 0.9918842 | 0.7618617 |
| 0.67 | 0.1335906 | 0.4382241 | -0.7253128 | 0.9924940 | 0.7604839 |
| 0.68 | 0.1343778 | 0.4381380 | -0.7243570 | 0.9931125 | 0.7590703 |
| 0.69 | 0.1351855 | 0.4380464 | -0.7233697 | 0.9937407 | 0.7576182 |
| 0.70 | 0.1360153 | 0.4379489 | -0.7223487 | 0.9943793 | 0.7561249 |
| 0.71 | 0.1368689 | 0.4378451 | -0.7212917 | 0.9950295 | 0.7545872 |
| 0.72 | 0.1377481 | 0.4377347 | -0.7201961 | 0.9956924 | 0.7530015 |
| 0.73 | 0.1386552 | 0.4376174 | -0.7190590 | 0.9963695 | 0.7513641 |
| 0.74 | 0.1395924 | 0.4374926 | -0.7178773 | 0.9970622 | 0.7496706 |
| 0.75 | 0.1405624 | 0.4373600 | -0.7166474 | 0.9977722 | 0.7479161 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.1224 Min. :0.4374 Min. :-0.7382
1st Qu.:0.5625 1st Qu.:0.1261 1st Qu.:0.4381 1st Qu.:-0.7340
Median :0.6250 Median :0.1303 Median :0.4385 Median :-0.7293
Mean :0.6250 Mean :0.1307 Mean :0.4384 Mean :-0.7286
3rd Qu.:0.6875 3rd Qu.:0.1350 3rd Qu.:0.4389 3rd Qu.:-0.7236
Max. :0.7500 Max. :0.1406 Max. :0.4390 Max. :-0.7166
ci.ub pvalue
Min. :0.9829 Min. :0.7479
1st Qu.:0.9863 1st Qu.:0.7580
Median :0.9898 Median :0.7664
Mean :0.9900 Mean :0.7657
3rd Qu.:0.9936 3rd Qu.:0.7738
Max. :0.9978 Max. :0.7805
Code
#less than .7
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$reliable_less <- ifelse(dataN33_prepost$reliability_measurement == 0, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = reliable_less)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue |
|---|---|---|---|---|---|
| 0.50 | -0.0586480 | 0.4999269 | -1.038487 | 0.9211907 | 0.9066120 |
| 0.51 | -0.0590084 | 0.5000127 | -1.039015 | 0.9209985 | 0.9060567 |
| 0.52 | -0.0593754 | 0.5000955 | -1.039545 | 0.9207937 | 0.9054907 |
| 0.53 | -0.0597492 | 0.5001750 | -1.040074 | 0.9205758 | 0.9049136 |
| 0.54 | -0.0601304 | 0.5002512 | -1.040605 | 0.9203440 | 0.9043245 |
| 0.55 | -0.0605192 | 0.5003239 | -1.041136 | 0.9200977 | 0.9037228 |
| 0.56 | -0.0609161 | 0.5003930 | -1.041668 | 0.9198361 | 0.9031077 |
| 0.57 | -0.0613217 | 0.5004583 | -1.042202 | 0.9195585 | 0.9024784 |
| 0.58 | -0.0617365 | 0.5005196 | -1.042737 | 0.9192639 | 0.9018340 |
| 0.59 | -0.0621611 | 0.5005768 | -1.043274 | 0.9189514 | 0.9011736 |
| 0.60 | -0.0625961 | 0.5006297 | -1.043812 | 0.9186200 | 0.9004961 |
| 0.61 | -0.0630421 | 0.5006780 | -1.044353 | 0.9182687 | 0.8998004 |
| 0.62 | -0.0635000 | 0.5007216 | -1.044896 | 0.9178963 | 0.8990853 |
| 0.63 | -0.0639704 | 0.5007602 | -1.045442 | 0.9175016 | 0.8983495 |
| 0.64 | -0.0644543 | 0.5007936 | -1.045992 | 0.9170832 | 0.8975916 |
| 0.65 | -0.0649526 | 0.5008216 | -1.046545 | 0.9166397 | 0.8968101 |
| 0.66 | -0.0654663 | 0.5008438 | -1.047102 | 0.9161695 | 0.8960031 |
| 0.67 | -0.0659966 | 0.5008600 | -1.047664 | 0.9156709 | 0.8951690 |
| 0.68 | -0.0665446 | 0.5008698 | -1.048231 | 0.9151421 | 0.8943055 |
| 0.69 | -0.0671118 | 0.5008729 | -1.048805 | 0.9145809 | 0.8934107 |
| 0.70 | -0.0676996 | 0.5008689 | -1.049385 | 0.9139853 | 0.8924819 |
| 0.71 | -0.0683096 | 0.5008574 | -1.049972 | 0.9133528 | 0.8915166 |
| 0.72 | -0.0689437 | 0.5008380 | -1.050568 | 0.9126808 | 0.8905117 |
| 0.73 | -0.0696038 | 0.5008103 | -1.051174 | 0.9119663 | 0.8894641 |
| 0.74 | -0.0702921 | 0.5007736 | -1.051790 | 0.9112062 | 0.8883700 |
| 0.75 | -0.0710111 | 0.5007276 | -1.052419 | 0.9103969 | 0.8872255 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.07101 Min. :0.4999 Min. :-1.052
1st Qu.:0.5625 1st Qu.:-0.06697 1st Qu.:0.5004 1st Qu.:-1.049
Median :0.6250 Median :-0.06374 Median :0.5007 Median :-1.045
Mean :0.6250 Mean :-0.06412 Mean :0.5006 Mean :-1.045
3rd Qu.:0.6875 3rd Qu.:-0.06102 3rd Qu.:0.5008 3rd Qu.:-1.042
Max. :0.7500 Max. :-0.05865 Max. :0.5009 Max. :-1.038
ci.ub pvalue
Min. :0.9104 Min. :0.8872
1st Qu.:0.9147 1st Qu.:0.8936
Median :0.9177 Median :0.8987
Mean :0.9170 Mean :0.8981
3rd Qu.:0.9198 3rd Qu.:0.9030
Max. :0.9212 Max. :0.9066
Code
#greater than .7
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$reliable_large <- ifelse(dataN33_prepost$reliability_measurement == 1, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = reliable_large)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue |
|---|---|---|---|---|---|
| 0.50 | -0.0521063 | 0.3609216 | -0.7594997 | 0.6552871 | 0.8852082 |
| 0.51 | -0.0523357 | 0.3609369 | -0.7597590 | 0.6550875 | 0.8847111 |
| 0.52 | -0.0525670 | 0.3609499 | -0.7600159 | 0.6548818 | 0.8842093 |
| 0.53 | -0.0528003 | 0.3609607 | -0.7602703 | 0.6546697 | 0.8837027 |
| 0.54 | -0.0530355 | 0.3609692 | -0.7605221 | 0.6544511 | 0.8831909 |
| 0.55 | -0.0532729 | 0.3609752 | -0.7607713 | 0.6542256 | 0.8826739 |
| 0.56 | -0.0535124 | 0.3609787 | -0.7610178 | 0.6539929 | 0.8821513 |
| 0.57 | -0.0537543 | 0.3609796 | -0.7612613 | 0.6537527 | 0.8816229 |
| 0.58 | -0.0539985 | 0.3609777 | -0.7615018 | 0.6535048 | 0.8810884 |
| 0.59 | -0.0542453 | 0.3609730 | -0.7617393 | 0.6532487 | 0.8805475 |
| 0.60 | -0.0544947 | 0.3609652 | -0.7619735 | 0.6529841 | 0.8799998 |
| 0.61 | -0.0547469 | 0.3609543 | -0.7622043 | 0.6527105 | 0.8794452 |
| 0.62 | -0.0550020 | 0.3609401 | -0.7624316 | 0.6524276 | 0.8788831 |
| 0.63 | -0.0552601 | 0.3609225 | -0.7626552 | 0.6521350 | 0.8783131 |
| 0.64 | -0.0555215 | 0.3609013 | -0.7628750 | 0.6518320 | 0.8777349 |
| 0.65 | -0.0557863 | 0.3608763 | -0.7630908 | 0.6515182 | 0.8771480 |
| 0.66 | -0.0560547 | 0.3608473 | -0.7633025 | 0.6511930 | 0.8765517 |
| 0.67 | -0.0563270 | 0.3608142 | -0.7635099 | 0.6508559 | 0.8759457 |
| 0.68 | -0.0566033 | 0.3607767 | -0.7637128 | 0.6505061 | 0.8753292 |
| 0.69 | -0.0568840 | 0.3607346 | -0.7639109 | 0.6501429 | 0.8747016 |
| 0.70 | -0.0571694 | 0.3606877 | -0.7641043 | 0.6497656 | 0.8740620 |
| 0.71 | -0.0574597 | 0.3606356 | -0.7642925 | 0.6493732 | 0.8734098 |
| 0.72 | -0.0577553 | 0.3605782 | -0.7644756 | 0.6489649 | 0.8727438 |
| 0.73 | -0.0580568 | 0.3605150 | -0.7646532 | 0.6485396 | 0.8720631 |
| 0.74 | -0.0583645 | 0.3604457 | -0.7648251 | 0.6480962 | 0.8713665 |
| 0.75 | -0.0586790 | 0.3603701 | -0.7649913 | 0.6476334 | 0.8706525 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.05868 Min. :0.3604 Min. :-0.7650
1st Qu.:0.5625 1st Qu.:-0.05681 1st Qu.:0.3607 1st Qu.:-0.7639
Median :0.6250 Median :-0.05513 Median :0.3609 Median :-0.7625
Mean :0.6250 Mean :-0.05522 Mean :0.3608 Mean :-0.7624
3rd Qu.:0.6875 3rd Qu.:-0.05357 3rd Qu.:0.3610 3rd Qu.:-0.7611
Max. :0.7500 Max. :-0.05211 Max. :0.3610 Max. :-0.7595
ci.ub pvalue
Min. :0.6476 Min. :0.8707
1st Qu.:0.6502 1st Qu.:0.8749
Median :0.6523 Median :0.8786
Mean :0.6520 Mean :0.8784
3rd Qu.:0.6539 3rd Qu.:0.8820
Max. :0.6553 Max. :0.8852
###Treatment fidelity
Code
##treatment_fidelity
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$fidelity <- as.factor(dataN33_prepost$treatment_fidelity)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = fidelity)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.0007552 | 0.1971821 | -0.3857147 | 0.3872250 | 0.9969443 | 0.4327666 |
| 0.51 | 0.0006399 | 0.1971904 | -0.3858462 | 0.3871260 | 0.9974109 | 0.4330532 |
| 0.52 | 0.0005233 | 0.1971975 | -0.3859766 | 0.3870233 | 0.9978825 | 0.4333346 |
| 0.53 | 0.0004055 | 0.1972033 | -0.3861059 | 0.3869168 | 0.9983595 | 0.4336106 |
| 0.54 | 0.0002862 | 0.1972079 | -0.3862341 | 0.3868065 | 0.9988422 | 0.4338809 |
| 0.55 | 0.0001653 | 0.1972111 | -0.3863613 | 0.3866919 | 0.9993311 | 0.4341453 |
| 0.56 | 0.0000429 | 0.1972129 | -0.3864873 | 0.3865730 | 0.9998265 | 0.4344036 |
| 0.57 | -0.0000813 | 0.1972133 | -0.3866122 | 0.3864496 | 0.9996710 | 0.4346555 |
| 0.58 | -0.0002074 | 0.1972121 | -0.3867360 | 0.3863212 | 0.9991610 | 0.4349008 |
| 0.59 | -0.0003354 | 0.1972094 | -0.3868587 | 0.3861878 | 0.9986429 | 0.4351390 |
| 0.60 | -0.0004656 | 0.1972050 | -0.3869803 | 0.3860491 | 0.9981161 | 0.4353700 |
| 0.61 | -0.0005981 | 0.1971989 | -0.3871008 | 0.3859046 | 0.9975800 | 0.4355933 |
| 0.62 | -0.0007331 | 0.1971910 | -0.3872203 | 0.3857541 | 0.9970338 | 0.4358086 |
| 0.63 | -0.0008707 | 0.1971812 | -0.3873387 | 0.3855973 | 0.9964768 | 0.4360156 |
| 0.64 | -0.0010111 | 0.1971694 | -0.3874561 | 0.3854338 | 0.9959082 | 0.4362138 |
| 0.65 | -0.0011547 | 0.1971556 | -0.3875725 | 0.3852632 | 0.9953271 | 0.4364028 |
| 0.66 | -0.0013015 | 0.1971396 | -0.3876880 | 0.3850850 | 0.9947324 | 0.4365822 |
| 0.67 | -0.0014519 | 0.1971213 | -0.3878026 | 0.3848987 | 0.9941231 | 0.4367514 |
| 0.68 | -0.0016062 | 0.1971006 | -0.3879164 | 0.3847039 | 0.9934979 | 0.4369100 |
| 0.69 | -0.0017647 | 0.1970774 | -0.3880294 | 0.3845000 | 0.9928556 | 0.4370573 |
| 0.70 | -0.0019277 | 0.1970516 | -0.3881417 | 0.3842863 | 0.9921947 | 0.4371928 |
| 0.71 | -0.0020956 | 0.1970229 | -0.3882535 | 0.3840622 | 0.9915135 | 0.4373157 |
| 0.72 | -0.0022689 | 0.1969913 | -0.3883648 | 0.3838270 | 0.9908103 | 0.4374255 |
| 0.73 | -0.0024480 | 0.1969566 | -0.3884759 | 0.3835798 | 0.9900831 | 0.4375214 |
| 0.74 | -0.0026335 | 0.1969186 | -0.3885868 | 0.3833198 | 0.9893298 | 0.4376024 |
| 0.75 | -0.0028259 | 0.1968770 | -0.3886978 | 0.3830460 | 0.9885477 | 0.4376678 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-2.826e-03 Min. :0.1969 Min. :-0.3887
1st Qu.:0.5625 1st Qu.:-1.725e-03 1st Qu.:0.1971 1st Qu.:-0.3880
Median :0.6250 Median :-8.019e-04 Median :0.1972 Median :-0.3873
Mean :0.6250 Mean :-8.832e-04 Mean :0.1971 Mean :-0.3873
3rd Qu.:0.6875 3rd Qu.: 1.183e-05 3rd Qu.:0.1972 3rd Qu.:-0.3865
Max. :0.7500 Max. : 7.552e-04 Max. :0.1972 Max. :-0.3857
ci.ub pvalue sigma2
Min. :0.3830 Min. :0.9885 Min. :0.4328
1st Qu.:0.3846 1st Qu.:0.9930 1st Qu.:0.4345
Median :0.3857 Median :0.9967 Median :0.4359
Mean :0.3855 Mean :0.9955 Mean :0.4357
3rd Qu.:0.3865 3rd Qu.:0.9983 3rd Qu.:0.4370
Max. :0.3872 Max. :0.9998 Max. :0.4377
Code
#not specified
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$fidelity_not <- ifelse(dataN33_prepost$treatment_fidelity == 0, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = fidelity_not)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue |
|---|---|---|---|---|---|
| 0.50 | 0.1532679 | 0.3602938 | -0.5528951 | 0.8594308 | 0.6705474 |
| 0.51 | 0.1534385 | 0.3603030 | -0.5527424 | 0.8596193 | 0.6702102 |
| 0.52 | 0.1536067 | 0.3603100 | -0.5525879 | 0.8598013 | 0.6698760 |
| 0.53 | 0.1537724 | 0.3603148 | -0.5524316 | 0.8599765 | 0.6695451 |
| 0.54 | 0.1539357 | 0.3603174 | -0.5522734 | 0.8601448 | 0.6692173 |
| 0.55 | 0.1540963 | 0.3603176 | -0.5521132 | 0.8603058 | 0.6688929 |
| 0.56 | 0.1542542 | 0.3603153 | -0.5519508 | 0.8604593 | 0.6685717 |
| 0.57 | 0.1544094 | 0.3603105 | -0.5517861 | 0.8606050 | 0.6682540 |
| 0.58 | 0.1545618 | 0.3603029 | -0.5516190 | 0.8607426 | 0.6679397 |
| 0.59 | 0.1547113 | 0.3602926 | -0.5514492 | 0.8608718 | 0.6676289 |
| 0.60 | 0.1548577 | 0.3602793 | -0.5512768 | 0.8609923 | 0.6673216 |
| 0.61 | 0.1550011 | 0.3602630 | -0.5511014 | 0.8611036 | 0.6670179 |
| 0.62 | 0.1551413 | 0.3602435 | -0.5509229 | 0.8612056 | 0.6667179 |
| 0.63 | 0.1552782 | 0.3602206 | -0.5507411 | 0.8612976 | 0.6664216 |
| 0.64 | 0.1554118 | 0.3601942 | -0.5505558 | 0.8613794 | 0.6661291 |
| 0.65 | 0.1555419 | 0.3601641 | -0.5503668 | 0.8614505 | 0.6658403 |
| 0.66 | 0.1556683 | 0.3601301 | -0.5501737 | 0.8615104 | 0.6655555 |
| 0.67 | 0.1557911 | 0.3600921 | -0.5499764 | 0.8615586 | 0.6652746 |
| 0.68 | 0.1559100 | 0.3600497 | -0.5497745 | 0.8615946 | 0.6649976 |
| 0.69 | 0.1560250 | 0.3600029 | -0.5495677 | 0.8616177 | 0.6647247 |
| 0.70 | 0.1561359 | 0.3599513 | -0.5493558 | 0.8616275 | 0.6644559 |
| 0.71 | 0.1562425 | 0.3598947 | -0.5491382 | 0.8616231 | 0.6641912 |
| 0.72 | 0.1563447 | 0.3598328 | -0.5489146 | 0.8616040 | 0.6639307 |
| 0.73 | 0.1564424 | 0.3597652 | -0.5486846 | 0.8615693 | 0.6636745 |
| 0.74 | 0.1565353 | 0.3596918 | -0.5484477 | 0.8615182 | 0.6634225 |
| 0.75 | 0.1566233 | 0.3596120 | -0.5482033 | 0.8614498 | 0.6631749 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.1533 Min. :0.3596 Min. :-0.5529
1st Qu.:0.5625 1st Qu.:0.1543 1st Qu.:0.3600 1st Qu.:-0.5519
Median :0.6250 Median :0.1552 Median :0.3602 Median :-0.5508
Mean :0.6250 Mean :0.1551 Mean :0.3601 Mean :-0.5507
3rd Qu.:0.6875 3rd Qu.:0.1560 3rd Qu.:0.3603 3rd Qu.:-0.5496
Max. :0.7500 Max. :0.1566 Max. :0.3603 Max. :-0.5482
ci.ub pvalue
Min. :0.8594 Min. :0.6632
1st Qu.:0.8605 1st Qu.:0.6648
Median :0.8613 Median :0.6666
Mean :0.8610 Mean :0.6667
3rd Qu.:0.8615 3rd Qu.:0.6685
Max. :0.8616 Max. :0.6705
Code
#partially reported
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$fidelity_part <- ifelse(dataN33_prepost$treatment_fidelity == 1, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = fidelity_part)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue |
|---|---|---|---|---|---|
| 0.50 | -0.3013057 | 0.3493059 | -0.9859326 | 0.3833213 | 0.3883662 |
| 0.51 | -0.3012633 | 0.3493043 | -0.9858873 | 0.3833606 | 0.3884308 |
| 0.52 | -0.3012121 | 0.3493012 | -0.9858298 | 0.3834055 | 0.3885071 |
| 0.53 | -0.3011517 | 0.3492962 | -0.9857598 | 0.3834563 | 0.3885956 |
| 0.54 | -0.3010816 | 0.3492895 | -0.9856765 | 0.3835132 | 0.3886968 |
| 0.55 | -0.3010014 | 0.3492809 | -0.9855795 | 0.3835766 | 0.3888115 |
| 0.56 | -0.3009106 | 0.3492704 | -0.9854680 | 0.3836467 | 0.3889403 |
| 0.57 | -0.3008087 | 0.3492578 | -0.9853415 | 0.3837240 | 0.3890839 |
| 0.58 | -0.3006951 | 0.3492431 | -0.9851991 | 0.3838088 | 0.3892431 |
| 0.59 | -0.3005693 | 0.3492263 | -0.9850402 | 0.3839016 | 0.3894187 |
| 0.60 | -0.3004305 | 0.3492071 | -0.9848638 | 0.3840028 | 0.3896116 |
| 0.61 | -0.3002782 | 0.3491855 | -0.9846692 | 0.3841128 | 0.3898227 |
| 0.62 | -0.3001115 | 0.3491615 | -0.9844554 | 0.3842323 | 0.3900532 |
| 0.63 | -0.2999298 | 0.3491348 | -0.9842214 | 0.3843618 | 0.3903042 |
| 0.64 | -0.2997321 | 0.3491054 | -0.9839661 | 0.3845019 | 0.3905767 |
| 0.65 | -0.2995175 | 0.3490731 | -0.9836883 | 0.3846533 | 0.3908723 |
| 0.66 | -0.2992850 | 0.3490379 | -0.9833867 | 0.3848167 | 0.3911924 |
| 0.67 | -0.2990335 | 0.3489995 | -0.9830600 | 0.3849930 | 0.3915384 |
| 0.68 | -0.2987619 | 0.3489579 | -0.9827068 | 0.3851829 | 0.3919122 |
| 0.69 | -0.2984689 | 0.3489128 | -0.9823253 | 0.3853876 | 0.3923156 |
| 0.70 | -0.2981530 | 0.3488641 | -0.9819139 | 0.3856080 | 0.3927508 |
| 0.71 | -0.2978126 | 0.3488115 | -0.9814707 | 0.3858454 | 0.3932200 |
| 0.72 | -0.2974462 | 0.3487550 | -0.9809935 | 0.3861011 | 0.3937258 |
| 0.73 | -0.2970518 | 0.3486943 | -0.9804801 | 0.3863764 | 0.3942709 |
| 0.74 | -0.2966274 | 0.3486291 | -0.9799279 | 0.3866731 | 0.3948586 |
| 0.75 | -0.2961706 | 0.3485593 | -0.9793343 | 0.3869930 | 0.3954922 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.3013 Min. :0.3486 Min. :-0.9859
1st Qu.:0.5625 1st Qu.:-0.3009 1st Qu.:0.3489 1st Qu.:-0.9854
Median :0.6250 Median :-0.3000 Median :0.3491 Median :-0.9843
Mean :0.6250 Mean :-0.2996 Mean :0.3491 Mean :-0.9837
3rd Qu.:0.6875 3rd Qu.:-0.2985 3rd Qu.:0.3493 3rd Qu.:-0.9824
Max. :0.7500 Max. :-0.2962 Max. :0.3493 Max. :-0.9793
ci.ub pvalue
Min. :0.3833 Min. :0.3884
1st Qu.:0.3837 1st Qu.:0.3890
Median :0.3843 Median :0.3902
Mean :0.3846 Mean :0.3908
3rd Qu.:0.3853 3rd Qu.:0.3922
Max. :0.3870 Max. :0.3955
Code
#lot of explanations
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$fidelity_yes <- ifelse(dataN33_prepost$treatment_fidelity == 2, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = fidelity_yes)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue |
|---|---|---|---|---|---|
| 0.50 | 0.1327394 | 0.3286621 | -0.5114264 | 0.7769052 | 0.6863024 |
| 0.51 | 0.1325507 | 0.3286750 | -0.5116404 | 0.7767417 | 0.6867363 |
| 0.52 | 0.1323563 | 0.3286860 | -0.5118564 | 0.7765690 | 0.6871814 |
| 0.53 | 0.1321559 | 0.3286951 | -0.5120747 | 0.7763865 | 0.6876382 |
| 0.54 | 0.1319492 | 0.3287022 | -0.5122952 | 0.7761937 | 0.6881073 |
| 0.55 | 0.1317359 | 0.3287072 | -0.5125183 | 0.7759902 | 0.6885896 |
| 0.56 | 0.1315157 | 0.3287100 | -0.5127442 | 0.7757755 | 0.6890856 |
| 0.57 | 0.1312880 | 0.3287106 | -0.5129730 | 0.7755490 | 0.6895963 |
| 0.58 | 0.1310525 | 0.3287088 | -0.5132049 | 0.7753100 | 0.6901225 |
| 0.59 | 0.1308088 | 0.3287046 | -0.5134403 | 0.7750580 | 0.6906651 |
| 0.60 | 0.1305564 | 0.3286978 | -0.5136795 | 0.7747923 | 0.6912252 |
| 0.61 | 0.1302948 | 0.3286884 | -0.5139226 | 0.7745122 | 0.6918038 |
| 0.62 | 0.1300234 | 0.3286761 | -0.5141700 | 0.7742167 | 0.6924022 |
| 0.63 | 0.1297415 | 0.3286610 | -0.5144222 | 0.7739052 | 0.6930216 |
| 0.64 | 0.1294486 | 0.3286428 | -0.5146794 | 0.7735766 | 0.6936634 |
| 0.65 | 0.1291439 | 0.3286214 | -0.5149422 | 0.7732300 | 0.6943292 |
| 0.66 | 0.1288266 | 0.3285967 | -0.5152110 | 0.7728643 | 0.6950206 |
| 0.67 | 0.1284960 | 0.3285684 | -0.5154863 | 0.7724783 | 0.6957394 |
| 0.68 | 0.1281510 | 0.3285366 | -0.5157688 | 0.7720708 | 0.6964877 |
| 0.69 | 0.1277906 | 0.3285008 | -0.5160591 | 0.7716404 | 0.6972676 |
| 0.70 | 0.1274138 | 0.3284610 | -0.5163579 | 0.7711855 | 0.6980815 |
| 0.71 | 0.1270193 | 0.3284169 | -0.5166661 | 0.7707046 | 0.6989322 |
| 0.72 | 0.1266057 | 0.3283684 | -0.5169845 | 0.7701959 | 0.6998225 |
| 0.73 | 0.1261716 | 0.3283151 | -0.5173141 | 0.7696574 | 0.7007558 |
| 0.74 | 0.1257153 | 0.3282568 | -0.5176562 | 0.7690868 | 0.7017356 |
| 0.75 | 0.1252350 | 0.3281932 | -0.5180119 | 0.7684818 | 0.7027661 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.1252 Min. :0.3282 Min. :-0.5180
1st Qu.:0.5625 1st Qu.:0.1279 1st Qu.:0.3285 1st Qu.:-0.5160
Median :0.6250 Median :0.1299 Median :0.3287 Median :-0.5143
Mean :0.6250 Mean :0.1296 Mean :0.3286 Mean :-0.5144
3rd Qu.:0.6875 3rd Qu.:0.1315 3rd Qu.:0.3287 3rd Qu.:-0.5128
Max. :0.7500 Max. :0.1327 Max. :0.3287 Max. :-0.5114
ci.ub pvalue
Min. :0.7685 Min. :0.6863
1st Qu.:0.7717 1st Qu.:0.6892
Median :0.7741 Median :0.6927
Mean :0.7736 Mean :0.6933
3rd Qu.:0.7757 3rd Qu.:0.6971
Max. :0.7769 Max. :0.7028
###Writing tasks
Code
##writing_tasks
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$tests <- as.factor(dataN33_prepost$writing_tests)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = tests)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.1864084 | 0.3376593 | -0.8482085 | 0.4753916 | 0.5809067 | 0.4274915 |
| 0.51 | -0.1865493 | 0.3376706 | -0.8483715 | 0.4752728 | 0.5806335 | 0.4277340 |
| 0.52 | -0.1866889 | 0.3376798 | -0.8485292 | 0.4751513 | 0.5803608 | 0.4279713 |
| 0.53 | -0.1868272 | 0.3376869 | -0.8486815 | 0.4750270 | 0.5800883 | 0.4282033 |
| 0.54 | -0.1869642 | 0.3376919 | -0.8488282 | 0.4748997 | 0.5798161 | 0.4284297 |
| 0.55 | -0.1871000 | 0.3376945 | -0.8489692 | 0.4747691 | 0.5795439 | 0.4286502 |
| 0.56 | -0.1872345 | 0.3376948 | -0.8491042 | 0.4746352 | 0.5792717 | 0.4288648 |
| 0.57 | -0.1873678 | 0.3376926 | -0.8492331 | 0.4744975 | 0.5789991 | 0.4290729 |
| 0.58 | -0.1874999 | 0.3376878 | -0.8493558 | 0.4743559 | 0.5787261 | 0.4292744 |
| 0.59 | -0.1876309 | 0.3376802 | -0.8494719 | 0.4742101 | 0.5784524 | 0.4294689 |
| 0.60 | -0.1877607 | 0.3376698 | -0.8495814 | 0.4740599 | 0.5781778 | 0.4296562 |
| 0.61 | -0.1878895 | 0.3376564 | -0.8496839 | 0.4739048 | 0.5779020 | 0.4298358 |
| 0.62 | -0.1880174 | 0.3376399 | -0.8497794 | 0.4737446 | 0.5776246 | 0.4300074 |
| 0.63 | -0.1881443 | 0.3376201 | -0.8498675 | 0.4735788 | 0.5773454 | 0.4301707 |
| 0.64 | -0.1882704 | 0.3375968 | -0.8499480 | 0.4734072 | 0.5770640 | 0.4303251 |
| 0.65 | -0.1883959 | 0.3375699 | -0.8500208 | 0.4732290 | 0.5767800 | 0.4304702 |
| 0.66 | -0.1885207 | 0.3375393 | -0.8500855 | 0.4730440 | 0.5764928 | 0.4306055 |
| 0.67 | -0.1886452 | 0.3375046 | -0.8501420 | 0.4728516 | 0.5762019 | 0.4307306 |
| 0.68 | -0.1887694 | 0.3374656 | -0.8501899 | 0.4726511 | 0.5759067 | 0.4308449 |
| 0.69 | -0.1888936 | 0.3374222 | -0.8502290 | 0.4724419 | 0.5756065 | 0.4309476 |
| 0.70 | -0.1890180 | 0.3373742 | -0.8502592 | 0.4722232 | 0.5753006 | 0.4310383 |
| 0.71 | -0.1891428 | 0.3373211 | -0.8502800 | 0.4719943 | 0.5749881 | 0.4311162 |
| 0.72 | -0.1892685 | 0.3372627 | -0.8502913 | 0.4717543 | 0.5746679 | 0.4311806 |
| 0.73 | -0.1893954 | 0.3371988 | -0.8502928 | 0.4715021 | 0.5743390 | 0.4312305 |
| 0.74 | -0.1895238 | 0.3371289 | -0.8502843 | 0.4712366 | 0.5740001 | 0.4312652 |
| 0.75 | -0.1896544 | 0.3370527 | -0.8502654 | 0.4709567 | 0.5736496 | 0.4312837 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.1897 Min. :0.3371 Min. :-0.8503
1st Qu.:0.5625 1st Qu.:-0.1889 1st Qu.:0.3374 1st Qu.:-0.8502
Median :0.6250 Median :-0.1881 Median :0.3376 Median :-0.8498
Mean :0.6250 Mean :-0.1881 Mean :0.3375 Mean :-0.8496
3rd Qu.:0.6875 3rd Qu.:-0.1873 3rd Qu.:0.3377 3rd Qu.:-0.8491
Max. :0.7500 Max. :-0.1864 Max. :0.3377 Max. :-0.8482
ci.ub pvalue sigma2
Min. :0.4710 Min. :0.5736 Min. :0.4275
1st Qu.:0.4725 1st Qu.:0.5757 1st Qu.:0.4289
Median :0.4737 Median :0.5775 Median :0.4301
Mean :0.4735 Mean :0.5774 Mean :0.4298
3rd Qu.:0.4746 3rd Qu.:0.5792 3rd Qu.:0.4309
Max. :0.4754 Max. :0.5809 Max. :0.4313
Code
#self-constructed
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$tests_self <- ifelse(dataN33_prepost$writing_tests == 0, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = tests_self)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue |
|---|---|---|---|---|---|
| 0.50 | 0.1864084 | 0.3376593 | -0.4753916 | 0.8482085 | 0.5809067 |
| 0.51 | 0.1865493 | 0.3376706 | -0.4752728 | 0.8483715 | 0.5806335 |
| 0.52 | 0.1866889 | 0.3376798 | -0.4751513 | 0.8485292 | 0.5803608 |
| 0.53 | 0.1868272 | 0.3376869 | -0.4750270 | 0.8486815 | 0.5800883 |
| 0.54 | 0.1869642 | 0.3376919 | -0.4748997 | 0.8488282 | 0.5798161 |
| 0.55 | 0.1871000 | 0.3376945 | -0.4747691 | 0.8489692 | 0.5795439 |
| 0.56 | 0.1872345 | 0.3376948 | -0.4746351 | 0.8491042 | 0.5792717 |
| 0.57 | 0.1873678 | 0.3376926 | -0.4744975 | 0.8492331 | 0.5789991 |
| 0.58 | 0.1874999 | 0.3376878 | -0.4743559 | 0.8493558 | 0.5787261 |
| 0.59 | 0.1876309 | 0.3376802 | -0.4742101 | 0.8494719 | 0.5784524 |
| 0.60 | 0.1877607 | 0.3376698 | -0.4740599 | 0.8495814 | 0.5781778 |
| 0.61 | 0.1878895 | 0.3376564 | -0.4739048 | 0.8496839 | 0.5779020 |
| 0.62 | 0.1880174 | 0.3376399 | -0.4737446 | 0.8497794 | 0.5776246 |
| 0.63 | 0.1881443 | 0.3376201 | -0.4735789 | 0.8498675 | 0.5773454 |
| 0.64 | 0.1882704 | 0.3375968 | -0.4734071 | 0.8499480 | 0.5770640 |
| 0.65 | 0.1883959 | 0.3375699 | -0.4732290 | 0.8500208 | 0.5767800 |
| 0.66 | 0.1885207 | 0.3375393 | -0.4730440 | 0.8500855 | 0.5764928 |
| 0.67 | 0.1886452 | 0.3375046 | -0.4728516 | 0.8501420 | 0.5762019 |
| 0.68 | 0.1887694 | 0.3374656 | -0.4726511 | 0.8501899 | 0.5759067 |
| 0.69 | 0.1888936 | 0.3374223 | -0.4724419 | 0.8502290 | 0.5756065 |
| 0.70 | 0.1890180 | 0.3373742 | -0.4722232 | 0.8502592 | 0.5753006 |
| 0.71 | 0.1891428 | 0.3373211 | -0.4719943 | 0.8502800 | 0.5749881 |
| 0.72 | 0.1892685 | 0.3372627 | -0.4717543 | 0.8502913 | 0.5746679 |
| 0.73 | 0.1893954 | 0.3371988 | -0.4715021 | 0.8502928 | 0.5743390 |
| 0.74 | 0.1895238 | 0.3371289 | -0.4712366 | 0.8502843 | 0.5740001 |
| 0.75 | 0.1896544 | 0.3370526 | -0.4709567 | 0.8502654 | 0.5736496 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.1864 Min. :0.3371 Min. :-0.4754
1st Qu.:0.5625 1st Qu.:0.1873 1st Qu.:0.3374 1st Qu.:-0.4746
Median :0.6250 Median :0.1881 Median :0.3376 Median :-0.4737
Mean :0.6250 Mean :0.1881 Mean :0.3375 Mean :-0.4735
3rd Qu.:0.6875 3rd Qu.:0.1889 3rd Qu.:0.3377 3rd Qu.:-0.4725
Max. :0.7500 Max. :0.1897 Max. :0.3377 Max. :-0.4710
ci.ub pvalue
Min. :0.8482 Min. :0.5736
1st Qu.:0.8491 1st Qu.:0.5757
Median :0.8498 Median :0.5775
Mean :0.8496 Mean :0.5774
3rd Qu.:0.8502 3rd Qu.:0.5792
Max. :0.8503 Max. :0.5809
Code
#standardized tests
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
data_pp_es <- escalc(measure = "SMCR",
m1i=data_pp$m_post, m2i=data_pp$m_pre,
sd1i=data_pp$sd_pre, sd2i=data_pp$sd_post,
ni=data_pp$ni, ri=rep(ri_t, 33),
slab = data_pp$slab)
# append the id variable
data_pp_es$id <- data_pp$id
data_pp_es$tests_norm <- ifelse(dataN33_prepost$writing_tests == 2, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=data_pp_es, random = ~ 1 | id, mods = tests_norm)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue |
|---|---|---|---|---|---|
| 0.50 | -0.1864084 | 0.3376593 | -0.8482085 | 0.4753916 | 0.5809067 |
| 0.51 | -0.1865493 | 0.3376706 | -0.8483715 | 0.4752728 | 0.5806335 |
| 0.52 | -0.1866889 | 0.3376798 | -0.8485292 | 0.4751513 | 0.5803608 |
| 0.53 | -0.1868272 | 0.3376869 | -0.8486815 | 0.4750270 | 0.5800883 |
| 0.54 | -0.1869642 | 0.3376919 | -0.8488282 | 0.4748997 | 0.5798161 |
| 0.55 | -0.1871000 | 0.3376945 | -0.8489692 | 0.4747691 | 0.5795439 |
| 0.56 | -0.1872345 | 0.3376948 | -0.8491042 | 0.4746351 | 0.5792717 |
| 0.57 | -0.1873678 | 0.3376926 | -0.8492331 | 0.4744975 | 0.5789991 |
| 0.58 | -0.1874999 | 0.3376878 | -0.8493558 | 0.4743559 | 0.5787261 |
| 0.59 | -0.1876309 | 0.3376802 | -0.8494719 | 0.4742102 | 0.5784524 |
| 0.60 | -0.1877607 | 0.3376698 | -0.8495814 | 0.4740599 | 0.5781778 |
| 0.61 | -0.1878895 | 0.3376564 | -0.8496839 | 0.4739048 | 0.5779020 |
| 0.62 | -0.1880174 | 0.3376399 | -0.8497794 | 0.4737446 | 0.5776246 |
| 0.63 | -0.1881443 | 0.3376201 | -0.8498675 | 0.4735788 | 0.5773454 |
| 0.64 | -0.1882704 | 0.3375968 | -0.8499480 | 0.4734071 | 0.5770640 |
| 0.65 | -0.1883959 | 0.3375699 | -0.8500208 | 0.4732290 | 0.5767800 |
| 0.66 | -0.1885207 | 0.3375393 | -0.8500855 | 0.4730440 | 0.5764928 |
| 0.67 | -0.1886452 | 0.3375046 | -0.8501420 | 0.4728516 | 0.5762019 |
| 0.68 | -0.1887694 | 0.3374656 | -0.8501899 | 0.4726511 | 0.5759067 |
| 0.69 | -0.1888936 | 0.3374223 | -0.8502290 | 0.4724419 | 0.5756065 |
| 0.70 | -0.1890180 | 0.3373742 | -0.8502592 | 0.4722232 | 0.5753006 |
| 0.71 | -0.1891428 | 0.3373211 | -0.8502800 | 0.4719943 | 0.5749881 |
| 0.72 | -0.1892685 | 0.3372627 | -0.8502913 | 0.4717543 | 0.5746679 |
| 0.73 | -0.1893954 | 0.3371988 | -0.8502928 | 0.4715021 | 0.5743390 |
| 0.74 | -0.1895238 | 0.3371289 | -0.8502843 | 0.4712366 | 0.5740001 |
| 0.75 | -0.1896544 | 0.3370526 | -0.8502654 | 0.4709567 | 0.5736496 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.1897 Min. :0.3371 Min. :-0.8503
1st Qu.:0.5625 1st Qu.:-0.1889 1st Qu.:0.3374 1st Qu.:-0.8502
Median :0.6250 Median :-0.1881 Median :0.3376 Median :-0.8498
Mean :0.6250 Mean :-0.1881 Mean :0.3375 Mean :-0.8496
3rd Qu.:0.6875 3rd Qu.:-0.1873 3rd Qu.:0.3377 3rd Qu.:-0.8491
Max. :0.7500 Max. :-0.1864 Max. :0.3377 Max. :-0.8482
ci.ub pvalue
Min. :0.4710 Min. :0.5736
1st Qu.:0.4725 1st Qu.:0.5757
Median :0.4737 Median :0.5775
Mean :0.4735 Mean :0.5774
3rd Qu.:0.4746 3rd Qu.:0.5792
Max. :0.4754 Max. :0.5809
Pre-Post-Control comparison
Code
# prepare data sets for meta-analysis
datT <- data.frame(
sampleNr = 1:14,
id = c(1,1,1,2,2,3,3,4,4,5,6,6,7,8),
m_pre = dataN14_control$M_Feedback,
m_post = dataN14_control$M_Feedback2,
sd_pre = dataN14_control$SD_Feedback,
sd_post = dataN14_control$SD_Feedback2,
ni = dataN14_control$N_Feedback,
slab = dataN14_control$author)
datC <- data.frame(
sampleNr = 1:14,
id = c(1,1,1,2,2,3,3,4,4,5,6,6,7,8),
m_pre = dataN14_control$M_NoFeedback,
m_post = dataN14_control$M_NoFeedback2,
sd_pre = dataN14_control$SD_NoFeedback,
sd_post = dataN14_control$SD_NoFeedback2,
ni = dataN14_control$N_noFeedback,
slab = dataN14_control$author)Meta-analytic effect
In this section, we used the same approach via sensitivity analyses as above.
Code
sensitivity <- data.frame(ri_t = as.numeric(), # assumed pre-post correlation treatment group
ri_c = as.numeric(), # assumed pre-post correlation control group
pvalue = as.numeric(), # p value of ES
beta = as.numeric(), # meta-analytic ES
se = as.numeric(), # SE of meta-analytic ES
sigma2 = as.numeric(), # tau squared
yi.f01 = as.numeric(), # ES of individual studies
yi.f02 = as.numeric(),
yi.f03 = as.numeric(),
yi.f04 = as.numeric(),
yi.f05 = as.numeric(),
yi.f06 = as.numeric(),
yi.f07 = as.numeric(),
yi.f08 = as.numeric(),
yi.f09 = as.numeric(),
yi.f10 = as.numeric(),
yi.f11 = as.numeric(),
yi.f12 = as.numeric(),
yi.f13 = as.numeric(),
yi.f14 = as.numeric(),
sei.f01 = as.numeric(), # SEs of ES of individual studies
sei.f02 = as.numeric(),
sei.f03 = as.numeric(),
sei.f04 = as.numeric(),
sei.f05 = as.numeric(),
sei.f06 = as.numeric(),
sei.f07 = as.numeric(),
sei.f08 = as.numeric(),
sei.f09 = as.numeric(),
sei.f10 = as.numeric(),
sei.f11 = as.numeric(),
sei.f12 = as.numeric(),
sei.f13 = as.numeric(),
sei.f14 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from between-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
rma_overall_clustered <- rma.mv(yi, vi, data=dat, random = ~ 1 | id)
sensitivity <- sensitivity %>%
add_row(ri_t = ri_t,
ri_c = (ri_t+.14),
pvalue = rma_overall_clustered$pval,
beta = rma_overall_clustered$beta[,1],
se = rma_overall_clustered$se[1],
sigma2 = rma_overall_clustered$sigma2[1],
yi.f01 = rma_overall_clustered$yi.f[1],
yi.f02 = rma_overall_clustered$yi.f[2],
yi.f03 = rma_overall_clustered$yi.f[3],
yi.f04 = rma_overall_clustered$yi.f[4],
yi.f05 = rma_overall_clustered$yi.f[5],
yi.f06 = rma_overall_clustered$yi.f[6],
yi.f07 = rma_overall_clustered$yi.f[7],
yi.f08 = rma_overall_clustered$yi.f[8],
yi.f09 = rma_overall_clustered$yi.f[9],
yi.f10 = rma_overall_clustered$yi.f[10],
yi.f11 = rma_overall_clustered$yi.f[11],
yi.f12 = rma_overall_clustered$yi.f[12],
yi.f13 = rma_overall_clustered$yi.f[13],
yi.f14 = rma_overall_clustered$yi.f[14],
sei.f01 = sqrt(dat$vi[1]),
sei.f02 = sqrt(dat$vi[2]),
sei.f03 = sqrt(dat$vi[3]),
sei.f04 = sqrt(dat$vi[4]),
sei.f05 = sqrt(dat$vi[5]),
sei.f06 = sqrt(dat$vi[6]),
sei.f07 = sqrt(dat$vi[7]),
sei.f08 = sqrt(dat$vi[8]),
sei.f09 = sqrt(dat$vi[9]),
sei.f10 = sqrt(dat$vi[10]),
sei.f11 = sqrt(dat$vi[11]),
sei.f12 = sqrt(dat$vi[12]),
sei.f13 = sqrt(dat$vi[13]),
sei.f14 = sqrt(dat$vi[14])
)
}Overview of meta-analytic ES
Results from all 26 meta-analyses:
- ri_t: assumed pre-post-correlation (treatment group)
- ri_c: assumed pre-post-correlation (control group)
- beta: meta-analytic ES
- se: SE of meta-analytic ES
- pvalue: p value of meta-analytic ES
Code
| ri_t | ri_c | beta | se | pvalue | sigma2 |
|---|---|---|---|---|---|
| 0.50 | 0.64 | 0.1224893 | 0.2240719 | 0.5846181 | 0.3377329 |
| 0.51 | 0.65 | 0.1224397 | 0.2237719 | 0.5842668 | 0.3374673 |
| 0.52 | 0.66 | 0.1223842 | 0.2234692 | 0.5839282 | 0.3371949 |
| 0.53 | 0.67 | 0.1223224 | 0.2231634 | 0.5836030 | 0.3369149 |
| 0.54 | 0.68 | 0.1222540 | 0.2228544 | 0.5832920 | 0.3366269 |
| 0.55 | 0.69 | 0.1221786 | 0.2225421 | 0.5829962 | 0.3363302 |
| 0.56 | 0.70 | 0.1220958 | 0.2222263 | 0.5827166 | 0.3360245 |
| 0.57 | 0.71 | 0.1220050 | 0.2219068 | 0.5824544 | 0.3357091 |
| 0.58 | 0.72 | 0.1219059 | 0.2215833 | 0.5822107 | 0.3353834 |
| 0.59 | 0.73 | 0.1217979 | 0.2212558 | 0.5819869 | 0.3350467 |
| 0.60 | 0.74 | 0.1216805 | 0.2209239 | 0.5817844 | 0.3346983 |
| 0.61 | 0.75 | 0.1215529 | 0.2205873 | 0.5816046 | 0.3343374 |
| 0.62 | 0.76 | 0.1214147 | 0.2202458 | 0.5814493 | 0.3339631 |
| 0.63 | 0.77 | 0.1212650 | 0.2198991 | 0.5813203 | 0.3335743 |
| 0.64 | 0.78 | 0.1211030 | 0.2195468 | 0.5812193 | 0.3331701 |
| 0.65 | 0.79 | 0.1209280 | 0.2191885 | 0.5811486 | 0.3327492 |
| 0.66 | 0.80 | 0.1207390 | 0.2188238 | 0.5811105 | 0.3323103 |
| 0.67 | 0.81 | 0.1205349 | 0.2184522 | 0.5811073 | 0.3318520 |
| 0.68 | 0.82 | 0.1203148 | 0.2180733 | 0.5811420 | 0.3313726 |
| 0.69 | 0.83 | 0.1200774 | 0.2176863 | 0.5812173 | 0.3308704 |
| 0.70 | 0.84 | 0.1198214 | 0.2172907 | 0.5813366 | 0.3303433 |
| 0.71 | 0.85 | 0.1195452 | 0.2168857 | 0.5815034 | 0.3297891 |
| 0.72 | 0.86 | 0.1192475 | 0.2164705 | 0.5817216 | 0.3292052 |
| 0.73 | 0.87 | 0.1189263 | 0.2160441 | 0.5819955 | 0.3285887 |
| 0.74 | 0.88 | 0.1185798 | 0.2156056 | 0.5823298 | 0.3279364 |
| 0.75 | 0.89 | 0.1182058 | 0.2151536 | 0.5827295 | 0.3272446 |
We summarize these results:
- ES_mean: meta-analytic ES (mean over the 26 meta-analyses)
- ES_min: meta-analytic ES (min of the 26 meta-analyses)
- ES_max: meta-analytic ES (max of the 26 meta-analyses)
- CI_lower_mean: lower CI of meta-analytic ES (mean over the 26 meta-analyses)
- CI_lower_min: lower CI of meta-analytic ES (min of the 26 meta-analyses)
- CI_lower_max: lower CI of meta-analytic ES (max of the 26 meta-analyses)
- CI_upper_mean: upper CI of meta-analytic ES (mean over the 26 meta-analyses)
- CI_upper_min: upper CI of meta-analytic ES (min of the 26 meta-analyses)
- CI_upper_max: upper CI of meta-analytic ES (max of the 26 meta-analyses)
- pvalue_mean: p value of meta-analytic ES (mean over the 26 meta-analyses)
- pvalue_min: p value of meta-analytic ES (min of the 26 meta-analyses)
- pvalue_max: p value of meta-analytic ES (max of the 26 meta-analyses)
Code
# compute mean, min and max of ES, CI and pvalue from meta-analysis
sensitivity %>%
dplyr::summarise(ES_mean = mean(beta),
ES_min = min(beta),
ES_max = max(beta),
CI_lower_mean = mean(beta-(1.96*se)),
CI_lower_min = min(beta-(1.96*se)),
CI_lower_max = max(beta-(1.96*se)),
CI_upper_mean = mean(beta+(1.96*se)),
CI_upper_min = min(beta+(1.96*se)),
CI_upper_max = max(beta+(1.96*se)),
pvalue_mean = mean(pvalue),
pvalue_min = min(pvalue),
pvalue_max = max(pvalue),
sigma2_mean = mean(sigma2)) %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover")) %>%
column_spec(1, background = "#C74A48") %>%
column_spec(2, background = "#B54341") %>%
column_spec(3, background = "#913634") %>%
column_spec(4, background = "#27A357") %>%
column_spec(5, background = "#208748") %>%
column_spec(6, background = "#186636") %>%
column_spec(7, background = "#B244B8") %>%
column_spec(8, background = "#8A358F") %>%
column_spec(9, background = "#6A296E") %>%
column_spec(10, background = "#9E8A47") %>%
column_spec(11, background = "#8F7C40") %>%
column_spec(12, background = "#6E6031")| ES_mean | ES_min | ES_max | CI_lower_mean | CI_lower_min | CI_lower_max | CI_upper_mean | CI_upper_min | CI_upper_max | pvalue_mean | pvalue_min | pvalue_max | sigma2_mean |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.1209927 | 0.1182058 | 0.1224893 | -0.3100356 | -0.3166917 | -0.3034952 | 0.5520209 | 0.5399069 | 0.5616703 | 0.5821843 | 0.5811073 | 0.5846181 | 0.3333245 |
We summarize the same aspects for all 22 studies:
Note that the ES does not vary due to different pre-post-correlation, but SE does.
Code
# compute mean, min and max of ES and CI from each study
sensitivity %>%
dplyr::select(-c(beta, pvalue, se, sigma2)) %>%
pivot_longer(c(3:30), # reshape data data from
names_to = "variable", # sensitivity analysis
values_to = "values") %>%
mutate(sampleNr = as.numeric(str_sub(variable, -2, -1)),
variable = str_sub(variable, 1, -5)) %>%
pivot_wider(id_cols = c(sampleNr, ri_t, ri_c),
names_from = "variable",
values_from = "values") %>%
group_by(sampleNr) %>%
dplyr::summarise(ES = mean(yi), # compute mean, min and max
CI_lower_mean = mean(yi-(1.96*sei)),
CI_lower_min = min(yi-(1.96*sei)),
CI_lower_max = max(yi-(1.96*sei)),
CI_upper_mean = mean(yi+(1.96*sei)),
CI_upper_min = min(yi+(1.96*sei)),
CI_upper_max = max(yi+(1.96*sei))) %>%
right_join(datT[c("sampleNr", "slab")],., by = "sampleNr") %>%
dplyr::select(-sampleNr) %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover")) %>%
column_spec(2, background = "#C74A48") %>%
column_spec(3, background = "#27A357") %>%
column_spec(4, background = "#208748") %>%
column_spec(5, background = "#186636") %>%
column_spec(6, background = "#B244B8") %>%
column_spec(7, background = "#8A358F") %>%
column_spec(8, background = "#6A296E")| slab | ES | CI_lower_mean | CI_lower_min | CI_lower_max | CI_upper_mean | CI_upper_min | CI_upper_max |
|---|---|---|---|---|---|---|---|
| Burkhart et al., 2020 | 0.6814068 | 0.0528333 | -0.0054387 | 0.1147960 | 1.3099802 | 1.2480176 | 1.3682523 |
| Burkhart et al., 2020 | -0.0702724 | -0.6009093 | -0.6721562 | -0.5229026 | 0.4603646 | 0.3823579 | 0.5316114 |
| Burkhart et al., 2020 | -0.1818085 | -0.6830836 | -0.7537560 | -0.6053296 | 0.3194665 | 0.2417125 | 0.3901389 |
| Frost, 2008 | -0.0504075 | -0.6613129 | -0.7785211 | -0.5273539 | 0.5604978 | 0.4265388 | 0.6777060 |
| Frost, 2008 | 0.6438224 | 0.0885290 | -0.0212781 | 0.2146052 | 1.1991159 | 1.0730397 | 1.3089229 |
| Kellogg et al., 2010 | -1.0744334 | -2.6117714 | -2.6436196 | -2.5794951 | 0.4629046 | 0.4306283 | 0.4947528 |
| Kellogg et al., 2010 | -0.8138863 | -2.3964347 | -2.4273843 | -2.3650927 | 0.7686620 | 0.7373200 | 0.7996117 |
| Lachner et al., 2017b (Study 2) | 0.8725517 | 0.2120163 | 0.1448799 | 0.2838472 | 1.5330871 | 1.4612562 | 1.6002236 |
| Lachner et al., 2017b (Study 3) | 0.4871669 | -0.1650553 | -0.2692787 | -0.0487959 | 1.1393891 | 1.0231297 | 1.2436125 |
| Lachner & Neuburg, 2019 | 0.3769218 | -0.1191822 | -0.1803689 | -0.0527118 | 0.8730258 | 0.8065553 | 0.9342125 |
| Palermo, 2017 | 0.6407101 | 0.5039751 | 0.4793736 | 0.5318357 | 0.7774452 | 0.7495846 | 0.8020467 |
| Palermo, 2017 | 1.1696352 | 1.0175403 | 0.9957871 | 1.0415086 | 1.3217301 | 1.2977617 | 1.3434833 |
| Roscoe et al., 2013 | -0.0231180 | -0.4029033 | -0.4765134 | -0.3186452 | 0.3566673 | 0.2724092 | 0.4302774 |
| Niloy et al., 2023 | -0.7949552 | -0.9304498 | -0.9529573 | -0.9052246 | -0.6594606 | -0.6846858 | -0.6369531 |
Forest Plot
In order to be able to display a forst plot, we calculated a meta-analysis with the mean assumed pre-post-correlation (r=.625 for the intervention group; r=.765 for the control group).
Code
## FOREST PLOT with mean correlation
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(.625, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((.625+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
REM <- rma.mv(yi, vi, data=dat, random = ~ 1 | id)
###FOREST PLOT
forest(REM)Bias estimation
Funnel plot
Code
##Funnel plot
funnel(REM, legend = T)Trim and fill
Code
Estimated number of missing studies on the right side: 1 (SE = 2.5068)
Random-Effects Model (k = 15; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0.3226 (SE = 0.1585)
tau (square root of estimated tau^2 value): 0.5679
I^2 (total heterogeneity / total variability): 92.54%
H^2 (total variability / sampling variability): 13.40
Test for Heterogeneity:
Q(df = 14) = 418.6696, p-val < .0001
Model Results:
estimate se zval pval ci.lb ci.ub
0.2687 0.1701 1.5796 0.1142 -0.0647 0.6022
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Code
funnel(taf_overall, legend = T)Egger’s test
Code
##Egger's test
regtest(rma_trimfill)
Regression Test for Funnel Plot Asymmetry
Model: mixed-effects meta-regression model
Predictor: standard error
Test for Funnel Plot Asymmetry: z = -1.2397, p = 0.2151
Limit Estimate (as sei -> 0): b = 0.5502 (CI: -0.0663, 1.1667)
Heterogeneity
Calculating \(I^2\)
Code
###HETEROGENEITY
# Establish empty data frame to be filled with results
heterogeneity_sen <- data.frame(ri_t = as.numeric(), # assumed pre-post correlation (treatment group)
ri_c = as.numeric(), # assumed pre-post correlation (control group)
I2 = as.numeric()) # I²
# starting loop over 26 possible pre-post-correlations
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from between-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
REM <- rma.mv(yi, vi,
data=dat,
random = ~ 1 | id # take clustered data into account
)
# Formula
W <- diag(1/REM$vi)
X <- model.matrix(REM)
P <- W-W%*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W
I2 <- 100*REM$sigma2/(sum(REM$sigma2)+(REM$k - REM$p)/sum(diag(P)))
# save estimates for sensitivity analysis
heterogeneity_sen <- heterogeneity_sen %>%
add_row(ri_t = ri_t,
ri_c = ri_t+.14,
I2 = I2)
}
heterogeneity_sen %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | ri_c | I2 |
|---|---|---|
| 0.50 | 0.64 | 91.28634 |
| 0.51 | 0.65 | 91.43733 |
| 0.52 | 0.66 | 91.58902 |
| 0.53 | 0.67 | 91.74141 |
| 0.54 | 0.68 | 91.89453 |
| 0.55 | 0.69 | 92.04837 |
| 0.56 | 0.70 | 92.20295 |
| 0.57 | 0.71 | 92.35829 |
| 0.58 | 0.72 | 92.51440 |
| 0.59 | 0.73 | 92.67129 |
| 0.60 | 0.74 | 92.82897 |
| 0.61 | 0.75 | 92.98747 |
| 0.62 | 0.76 | 93.14681 |
| 0.63 | 0.77 | 93.30699 |
| 0.64 | 0.78 | 93.46806 |
| 0.65 | 0.79 | 93.63001 |
| 0.66 | 0.80 | 93.79289 |
| 0.67 | 0.81 | 93.95672 |
| 0.68 | 0.82 | 94.12153 |
| 0.69 | 0.83 | 94.28736 |
| 0.70 | 0.84 | 94.45424 |
| 0.71 | 0.85 | 94.62221 |
| 0.72 | 0.86 | 94.79133 |
| 0.73 | 0.87 | 94.96165 |
| 0.74 | 0.88 | 95.13322 |
| 0.75 | 0.89 | 95.30613 |
Code
skim(heterogeneity_sen)| Name | heterogeneity_sen |
| Number of rows | 26 |
| Number of columns | 3 |
| _______________________ | |
| Column type frequency: | |
| numeric | 3 |
| ________________________ | |
| Group variables | None |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| ri_t | 0 | 1 | 0.625 | 0.076 | 0.500 | 0.562 | 0.625 | 0.688 | 0.750 | ▇▇▇▇▇ |
| ri_c | 0 | 1 | 0.765 | 0.076 | 0.640 | 0.703 | 0.765 | 0.827 | 0.890 | ▇▇▇▇▇ |
| I2 | 0 | 1 | 93.252 | 1.228 | 91.286 | 92.242 | 93.227 | 94.246 | 95.306 | ▇▇▇▇▇ |
Moderators
We will report results from all 26 meta-analyses for the moderators.
For each moderator we provide
- a table with the main parameters from each meta-analysis
- ri_t: assumed pre-post-correlation (sensitivity analysis)
- beta: ES of moderator
- ci.lb: lower bound CI of ES of moderator
- ci.ub: upper bound CI of ES of moderator
- pvalue: p value of ES of moderator
- a summary table of these parameters
Representation
Graphical representation
Code
#graphical representation
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$rep_g <- as.factor(dataN14_control$rep_graphical)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = rep_g)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.4645432 | 0.2991202 | -0.1217217 | 1.0508081 | 0.1204156 | 0.3155087 |
| 0.51 | 0.4636580 | 0.2972652 | -0.1189711 | 1.0462871 | 0.1188201 | 0.3149844 |
| 0.52 | 0.4627455 | 0.2953844 | -0.1161973 | 1.0416883 | 0.1172112 | 0.3144511 |
| 0.53 | 0.4618045 | 0.2934769 | -0.1133998 | 1.0370087 | 0.1155885 | 0.3139082 |
| 0.54 | 0.4608336 | 0.2915420 | -0.1105783 | 1.0322455 | 0.1139520 | 0.3133559 |
| 0.55 | 0.4598314 | 0.2895787 | -0.1077324 | 1.0273952 | 0.1123016 | 0.3127934 |
| 0.56 | 0.4587964 | 0.2875859 | -0.1048616 | 1.0224545 | 0.1106369 | 0.3122202 |
| 0.57 | 0.4577270 | 0.2855627 | -0.1019657 | 1.0174196 | 0.1089579 | 0.3116357 |
| 0.58 | 0.4566213 | 0.2835080 | -0.0990441 | 1.0122867 | 0.1072644 | 0.3110394 |
| 0.59 | 0.4554775 | 0.2814205 | -0.0960966 | 1.0070516 | 0.1055561 | 0.3104306 |
| 0.60 | 0.4542937 | 0.2792992 | -0.0931227 | 1.0017101 | 0.1038331 | 0.3098085 |
| 0.61 | 0.4530677 | 0.2771427 | -0.0901221 | 0.9962574 | 0.1020950 | 0.3091725 |
| 0.62 | 0.4517971 | 0.2749497 | -0.0870944 | 0.9906886 | 0.1003417 | 0.3085216 |
| 0.63 | 0.4504796 | 0.2727187 | -0.0840391 | 0.9849984 | 0.0985731 | 0.3078551 |
| 0.64 | 0.4491125 | 0.2704481 | -0.0809561 | 0.9791811 | 0.0967891 | 0.3071718 |
| 0.65 | 0.4476929 | 0.2681365 | -0.0778449 | 0.9732307 | 0.0949895 | 0.3064707 |
| 0.66 | 0.4462178 | 0.2657820 | -0.0747053 | 0.9671408 | 0.0931742 | 0.3057506 |
| 0.67 | 0.4446837 | 0.2633827 | -0.0715369 | 0.9609044 | 0.0913430 | 0.3050101 |
| 0.68 | 0.4430872 | 0.2609368 | -0.0683396 | 0.9545141 | 0.0894958 | 0.3042478 |
| 0.69 | 0.4414244 | 0.2584422 | -0.0651130 | 0.9479617 | 0.0876327 | 0.3034619 |
| 0.70 | 0.4396909 | 0.2558965 | -0.0618570 | 0.9412388 | 0.0857533 | 0.3026508 |
| 0.71 | 0.4378822 | 0.2532973 | -0.0585715 | 0.9343359 | 0.0838578 | 0.3018122 |
| 0.72 | 0.4359933 | 0.2506422 | -0.0552563 | 0.9272429 | 0.0819460 | 0.3009440 |
| 0.73 | 0.4340187 | 0.2479282 | -0.0519116 | 0.9199490 | 0.0800179 | 0.3000434 |
| 0.74 | 0.4319524 | 0.2451523 | -0.0485372 | 0.9124421 | 0.0780734 | 0.2991076 |
| 0.75 | 0.4297879 | 0.2423113 | -0.0451335 | 0.9047094 | 0.0761125 | 0.2981333 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.4298 Min. :0.2423 Min. :-0.12172
1st Qu.:0.5625 1st Qu.:0.4418 1st Qu.:0.2591 1st Qu.:-0.10414
Median :0.6250 Median :0.4511 Median :0.2738 Median :-0.08557
Mean :0.6250 Mean :0.4497 Mean :0.2727 Mean :-0.08480
3rd Qu.:0.6875 3rd Qu.:0.4585 3rd Qu.:0.2871 3rd Qu.:-0.06592
Max. :0.7500 Max. :0.4645 Max. :0.2991 Max. :-0.04513
ci.ub pvalue sigma2
Min. :0.9047 Min. :0.07611 Min. :0.2981
1st Qu.:0.9496 1st Qu.:0.08810 1st Qu.:0.3037
Median :0.9878 Median :0.09946 Median :0.3082
Mean :0.9843 Mean :0.09903 Mean :0.3077
3rd Qu.:1.0212 3rd Qu.:0.11022 3rd Qu.:0.3121
Max. :1.0508 Max. :0.12042 Max. :0.3155
Numerical representation
Code
#numeric representation
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$rep_n <- as.factor(dataN14_control$rep_numeric)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = rep_n)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.1396910 | 0.4739078 | -0.7891512 | 1.068533 | 0.7681741 | 0.3831438 |
| 0.51 | 0.1391449 | 0.4734058 | -0.7887135 | 1.067003 | 0.7688168 | 0.3830100 |
| 0.52 | 0.1386065 | 0.4728972 | -0.7882550 | 1.065468 | 0.7694452 | 0.3828662 |
| 0.53 | 0.1380763 | 0.4723817 | -0.7877748 | 1.063927 | 0.7700586 | 0.3827121 |
| 0.54 | 0.1375548 | 0.4718589 | -0.7872717 | 1.062381 | 0.7706562 | 0.3825470 |
| 0.55 | 0.1370422 | 0.4713284 | -0.7867446 | 1.060829 | 0.7712370 | 0.3823702 |
| 0.56 | 0.1365390 | 0.4707899 | -0.7861922 | 1.059270 | 0.7718001 | 0.3821811 |
| 0.57 | 0.1360458 | 0.4702428 | -0.7856132 | 1.057705 | 0.7723443 | 0.3819790 |
| 0.58 | 0.1355631 | 0.4696868 | -0.7850061 | 1.056132 | 0.7728687 | 0.3817629 |
| 0.59 | 0.1350914 | 0.4691212 | -0.7843693 | 1.054552 | 0.7733721 | 0.3815322 |
| 0.60 | 0.1346313 | 0.4685455 | -0.7837011 | 1.052964 | 0.7738530 | 0.3812856 |
| 0.61 | 0.1341834 | 0.4679591 | -0.7829997 | 1.051366 | 0.7743101 | 0.3810223 |
| 0.62 | 0.1337484 | 0.4673613 | -0.7822630 | 1.049760 | 0.7747420 | 0.3807411 |
| 0.63 | 0.1333270 | 0.4667514 | -0.7814889 | 1.048143 | 0.7751471 | 0.3804407 |
| 0.64 | 0.1329201 | 0.4661285 | -0.7806750 | 1.046515 | 0.7755234 | 0.3801198 |
| 0.65 | 0.1325283 | 0.4654917 | -0.7798186 | 1.044875 | 0.7758693 | 0.3797768 |
| 0.66 | 0.1321527 | 0.4648400 | -0.7789168 | 1.043222 | 0.7761826 | 0.3794100 |
| 0.67 | 0.1317943 | 0.4641723 | -0.7779667 | 1.041555 | 0.7764610 | 0.3790175 |
| 0.68 | 0.1314540 | 0.4634874 | -0.7769646 | 1.039872 | 0.7767022 | 0.3785974 |
| 0.69 | 0.1311330 | 0.4627838 | -0.7759067 | 1.038173 | 0.7769033 | 0.3781473 |
| 0.70 | 0.1308325 | 0.4620602 | -0.7747888 | 1.036454 | 0.7770615 | 0.3776646 |
| 0.71 | 0.1305540 | 0.4613147 | -0.7736061 | 1.034714 | 0.7771735 | 0.3771465 |
| 0.72 | 0.1302990 | 0.4605454 | -0.7723534 | 1.032951 | 0.7772357 | 0.3765897 |
| 0.73 | 0.1300690 | 0.4597501 | -0.7710247 | 1.031163 | 0.7772440 | 0.3759906 |
| 0.74 | 0.1298659 | 0.4589264 | -0.7696133 | 1.029345 | 0.7771940 | 0.3753451 |
| 0.75 | 0.1296919 | 0.4580715 | -0.7681117 | 1.027495 | 0.7770804 | 0.3746485 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.1297 Min. :0.4581 Min. :-0.7892
1st Qu.:0.5625 1st Qu.:0.1312 1st Qu.:0.4630 1st Qu.:-0.7860
Median :0.6250 Median :0.1335 Median :0.4671 Median :-0.7819
Mean :0.6250 Mean :0.1339 Mean :0.4667 Mean :-0.7807
3rd Qu.:0.6875 3rd Qu.:0.1364 3rd Qu.:0.4707 3rd Qu.:-0.7762
Max. :0.7500 Max. :0.1397 Max. :0.4739 Max. :-0.7681
ci.ub pvalue sigma2
Min. :1.027 Min. :0.7682 Min. :0.3746
1st Qu.:1.039 1st Qu.:0.7719 1st Qu.:0.3783
Median :1.049 Median :0.7749 Median :0.3806
Mean :1.049 Mean :0.7741 Mean :0.3800
3rd Qu.:1.059 3rd Qu.:0.7769 3rd Qu.:0.3821
Max. :1.069 Max. :0.7772 Max. :0.3831
Highlighting representation
Code
#highlighting representation
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$rep_h <- as.factor(dataN14_control$rep_highlighting)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = rep_h)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.2321299 | 0.3242949 | -0.4034764 | 0.8677362 | 0.4741155 | 0.3501718 |
| 0.51 | 0.2313462 | 0.3221528 | -0.4000617 | 0.8627540 | 0.4726798 | 0.3497470 |
| 0.52 | 0.2305327 | 0.3199823 | -0.3966212 | 0.8576865 | 0.4712453 | 0.3493126 |
| 0.53 | 0.2296878 | 0.3177827 | -0.3931549 | 0.8525305 | 0.4698135 | 0.3488678 |
| 0.54 | 0.2288100 | 0.3155529 | -0.3896624 | 0.8472824 | 0.4683857 | 0.3484124 |
| 0.55 | 0.2278974 | 0.3132921 | -0.3861437 | 0.8419386 | 0.4669638 | 0.3479456 |
| 0.56 | 0.2269483 | 0.3109991 | -0.3825987 | 0.8364953 | 0.4655495 | 0.3474669 |
| 0.57 | 0.2259605 | 0.3086729 | -0.3790273 | 0.8309483 | 0.4641449 | 0.3469756 |
| 0.58 | 0.2249321 | 0.3063124 | -0.3754293 | 0.8252934 | 0.4627523 | 0.3464710 |
| 0.59 | 0.2238605 | 0.3039165 | -0.3718049 | 0.8195259 | 0.4613744 | 0.3459523 |
| 0.60 | 0.2227435 | 0.3014838 | -0.3681539 | 0.8136410 | 0.4600138 | 0.3454188 |
| 0.61 | 0.2215784 | 0.2990131 | -0.3644766 | 0.8076333 | 0.4586738 | 0.3448695 |
| 0.62 | 0.2203622 | 0.2965030 | -0.3607729 | 0.8014974 | 0.4573581 | 0.3443034 |
| 0.63 | 0.2190920 | 0.2939519 | -0.3570430 | 0.7952271 | 0.4560704 | 0.3437190 |
| 0.64 | 0.2177645 | 0.2913583 | -0.3532874 | 0.7888163 | 0.4548154 | 0.3431160 |
| 0.65 | 0.2163760 | 0.2887207 | -0.3495062 | 0.7822581 | 0.4535980 | 0.3424928 |
| 0.66 | 0.2149227 | 0.2860371 | -0.3456998 | 0.7755452 | 0.4524238 | 0.3418478 |
| 0.67 | 0.2134004 | 0.2833059 | -0.3418689 | 0.7686697 | 0.4512991 | 0.3411795 |
| 0.68 | 0.2118047 | 0.2805248 | -0.3380139 | 0.7616232 | 0.4502310 | 0.3404863 |
| 0.69 | 0.2101305 | 0.2776919 | -0.3341357 | 0.7543967 | 0.4492275 | 0.3397661 |
| 0.70 | 0.2083725 | 0.2748049 | -0.3302351 | 0.7469802 | 0.4482976 | 0.3390168 |
| 0.71 | 0.2065250 | 0.2718612 | -0.3263133 | 0.7393632 | 0.4474515 | 0.3382362 |
| 0.72 | 0.2045814 | 0.2688584 | -0.3223713 | 0.7315342 | 0.4467011 | 0.3374214 |
| 0.73 | 0.2025349 | 0.2657935 | -0.3184108 | 0.7234806 | 0.4460593 | 0.3365697 |
| 0.74 | 0.2003779 | 0.2626636 | -0.3144333 | 0.7151890 | 0.4455416 | 0.3356775 |
| 0.75 | 0.1981019 | 0.2594653 | -0.3104408 | 0.7066446 | 0.4451651 | 0.3347412 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.1981 Min. :0.2595 Min. :-0.4035
1st Qu.:0.5625 1st Qu.:0.2105 1st Qu.:0.2784 1st Qu.:-0.3817
Median :0.6250 Median :0.2197 Median :0.2952 Median :-0.3589
Mean :0.6250 Mean :0.2181 Mean :0.2940 Mean :-0.3582
3rd Qu.:0.6875 3rd Qu.:0.2267 3rd Qu.:0.3104 3rd Qu.:-0.3351
Max. :0.7500 Max. :0.2321 Max. :0.3243 Max. :-0.3104
ci.ub pvalue sigma2
Min. :0.7066 Min. :0.4452 Min. :0.3347
1st Qu.:0.7562 1st Qu.:0.4495 1st Qu.:0.3399
Median :0.7984 Median :0.4567 Median :0.3440
Mean :0.7944 Mean :0.4577 Mean :0.3435
3rd Qu.:0.8351 3rd Qu.:0.4652 3rd Qu.:0.3473
Max. :0.8677 Max. :0.4741 Max. :0.3502
Text-based representation
Code
#text-based representation
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$rep_t <- as.factor(dataN14_control$rep_text_based)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = rep_t)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.1592562 | 0.4844770 | -1.108814 | 0.7903013 | 0.7423690 | 0.3909857 |
| 0.51 | -0.1594807 | 0.4838907 | -1.107889 | 0.7889277 | 0.7417173 | 0.3906129 |
| 0.52 | -0.1597059 | 0.4832982 | -1.106953 | 0.7875411 | 0.7410599 | 0.3902311 |
| 0.53 | -0.1599319 | 0.4826993 | -1.106005 | 0.7861413 | 0.7403966 | 0.3898403 |
| 0.54 | -0.1601588 | 0.4820931 | -1.105044 | 0.7847262 | 0.7397265 | 0.3894389 |
| 0.55 | -0.1603869 | 0.4814795 | -1.104069 | 0.7832955 | 0.7390493 | 0.3890268 |
| 0.56 | -0.1606163 | 0.4808581 | -1.103081 | 0.7818483 | 0.7383644 | 0.3886032 |
| 0.57 | -0.1608472 | 0.4802286 | -1.102078 | 0.7803835 | 0.7376712 | 0.3881676 |
| 0.58 | -0.1610799 | 0.4795903 | -1.101060 | 0.7788997 | 0.7369690 | 0.3877191 |
| 0.59 | -0.1613147 | 0.4789427 | -1.100025 | 0.7773958 | 0.7362570 | 0.3872568 |
| 0.60 | -0.1615519 | 0.4782854 | -1.098974 | 0.7758702 | 0.7355342 | 0.3867798 |
| 0.61 | -0.1617920 | 0.4776176 | -1.097905 | 0.7743212 | 0.7347995 | 0.3862871 |
| 0.62 | -0.1620354 | 0.4769386 | -1.096818 | 0.7727470 | 0.7340519 | 0.3857775 |
| 0.63 | -0.1622826 | 0.4762476 | -1.095711 | 0.7711456 | 0.7332899 | 0.3852498 |
| 0.64 | -0.1625343 | 0.4755439 | -1.094583 | 0.7695145 | 0.7325118 | 0.3847025 |
| 0.65 | -0.1627912 | 0.4748264 | -1.093434 | 0.7678515 | 0.7317161 | 0.3841342 |
| 0.66 | -0.1630542 | 0.4740941 | -1.092262 | 0.7661532 | 0.7309005 | 0.3835431 |
| 0.67 | -0.1633243 | 0.4733458 | -1.091065 | 0.7644165 | 0.7300626 | 0.3829273 |
| 0.68 | -0.1636028 | 0.4725803 | -1.089843 | 0.7626376 | 0.7291996 | 0.3822847 |
| 0.69 | -0.1638910 | 0.4717960 | -1.088594 | 0.7608121 | 0.7283083 | 0.3816129 |
| 0.70 | -0.1641906 | 0.4709912 | -1.087316 | 0.7589353 | 0.7273848 | 0.3809092 |
| 0.71 | -0.1645036 | 0.4701642 | -1.086008 | 0.7570014 | 0.7264248 | 0.3801706 |
| 0.72 | -0.1648324 | 0.4693128 | -1.084669 | 0.7550038 | 0.7254228 | 0.3793937 |
| 0.73 | -0.1651798 | 0.4684345 | -1.083295 | 0.7529350 | 0.7243726 | 0.3785745 |
| 0.74 | -0.1655494 | 0.4675267 | -1.081885 | 0.7507860 | 0.7232668 | 0.3777087 |
| 0.75 | -0.1659453 | 0.4665861 | -1.080437 | 0.7485465 | 0.7220963 | 0.3767909 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.1659 Min. :0.4666 Min. :-1.109
1st Qu.:0.5625 1st Qu.:-0.1638 1st Qu.:0.4720 1st Qu.:-1.103
Median :0.6250 Median :-0.1622 Median :0.4766 Median :-1.096
Mean :0.6250 Mean :-0.1623 Mean :0.4762 Mean :-1.096
3rd Qu.:0.6875 3rd Qu.:-0.1607 3rd Qu.:0.4807 3rd Qu.:-1.089
Max. :0.7500 Max. :-0.1593 Max. :0.4845 Max. :-1.080
ci.ub pvalue sigma2
Min. :0.7485 Min. :0.7221 Min. :0.3768
1st Qu.:0.7613 1st Qu.:0.7285 1st Qu.:0.3818
Median :0.7719 Median :0.7337 Median :0.3855
Mean :0.7711 Mean :0.7332 Mean :0.3850
3rd Qu.:0.7815 3rd Qu.:0.7382 3rd Qu.:0.3885
Max. :0.7903 Max. :0.7424 Max. :0.3910
Multiple representation formats
Code
#number of representations
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$rep_nr <- as.factor(dataN14_control$rep_nr)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = rep_nr)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.3724921 | 0.2900898 | -0.1960734 | 0.9410576 | 0.1991216 | 0.2900436 |
| 0.51 | 0.3700428 | 0.2885467 | -0.1954984 | 0.9355840 | 0.1996897 | 0.2902134 |
| 0.52 | 0.3675557 | 0.2869769 | -0.1949086 | 0.9300201 | 0.2002691 | 0.2903765 |
| 0.53 | 0.3650292 | 0.2853794 | -0.1943040 | 0.9243625 | 0.2008612 | 0.2905325 |
| 0.54 | 0.3624613 | 0.2837532 | -0.1936848 | 0.9186074 | 0.2014673 | 0.2906811 |
| 0.55 | 0.3598499 | 0.2820975 | -0.1930511 | 0.9127510 | 0.2020890 | 0.2908221 |
| 0.56 | 0.3571929 | 0.2804113 | -0.1924031 | 0.9067890 | 0.2027280 | 0.2909550 |
| 0.57 | 0.3544879 | 0.2786934 | -0.1917412 | 0.9007170 | 0.2033860 | 0.2910794 |
| 0.58 | 0.3517324 | 0.2769428 | -0.1910655 | 0.8945303 | 0.2040653 | 0.2911950 |
| 0.59 | 0.3489237 | 0.2751583 | -0.1903766 | 0.8882240 | 0.2047681 | 0.2913012 |
| 0.60 | 0.3460590 | 0.2733386 | -0.1896749 | 0.8817928 | 0.2054970 | 0.2913977 |
| 0.61 | 0.3431351 | 0.2714825 | -0.1889608 | 0.8752309 | 0.2062548 | 0.2914838 |
| 0.62 | 0.3401487 | 0.2695885 | -0.1882350 | 0.8685324 | 0.2070448 | 0.2915591 |
| 0.63 | 0.3370963 | 0.2676551 | -0.1874981 | 0.8616907 | 0.2078705 | 0.2916228 |
| 0.64 | 0.3339740 | 0.2656809 | -0.1867510 | 0.8546989 | 0.2087358 | 0.2916744 |
| 0.65 | 0.3307775 | 0.2636641 | -0.1859946 | 0.8475496 | 0.2096453 | 0.2917131 |
| 0.66 | 0.3275025 | 0.2616030 | -0.1852299 | 0.8402349 | 0.2106039 | 0.2917381 |
| 0.67 | 0.3241439 | 0.2594957 | -0.1844582 | 0.8327461 | 0.2116173 | 0.2917485 |
| 0.68 | 0.3206966 | 0.2573401 | -0.1836808 | 0.8250740 | 0.2126920 | 0.2917433 |
| 0.69 | 0.3171546 | 0.2551343 | -0.1828994 | 0.8172086 | 0.2138350 | 0.2917216 |
| 0.70 | 0.3135118 | 0.2528758 | -0.1821156 | 0.8091391 | 0.2150546 | 0.2916819 |
| 0.71 | 0.3097611 | 0.2505621 | -0.1813316 | 0.8008539 | 0.2163602 | 0.2916232 |
| 0.72 | 0.3058952 | 0.2481908 | -0.1805498 | 0.7923401 | 0.2177623 | 0.2915438 |
| 0.73 | 0.3019056 | 0.2457588 | -0.1797728 | 0.7835840 | 0.2192732 | 0.2914421 |
| 0.74 | 0.2977833 | 0.2432632 | -0.1790037 | 0.7745704 | 0.2209069 | 0.2913163 |
| 0.75 | 0.2935184 | 0.2407006 | -0.1782461 | 0.7652828 | 0.2226796 | 0.2911643 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.2935 Min. :0.2407 Min. :-0.1961
1st Qu.:0.5625 1st Qu.:0.3180 1st Qu.:0.2557 1st Qu.:-0.1922
Median :0.6250 Median :0.3386 Median :0.2686 Median :-0.1879
Mean :0.6250 Mean :0.3366 Mean :0.2675 Mean :-0.1876
3rd Qu.:0.6875 3rd Qu.:0.3565 3rd Qu.:0.2800 3rd Qu.:-0.1831
Max. :0.7500 Max. :0.3725 Max. :0.2901 Max. :-0.1782
ci.ub pvalue sigma2
Min. :0.7653 Min. :0.1991 Min. :0.2900
1st Qu.:0.8192 1st Qu.:0.2029 1st Qu.:0.2910
Median :0.8651 Median :0.2075 Median :0.2914
Mean :0.8609 Mean :0.2086 Mean :0.2912
3rd Qu.:0.9053 3rd Qu.:0.2135 3rd Qu.:0.2917
Max. :0.9411 Max. :0.2227 Max. :0.2917
Level of feedback
Code
## LEVEL ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$FB_order <- as.factor(dataN14_control$FB_order)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = FB_order)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.5200935 | 0.1157803 | -0.7470188 | -0.2931682 | 7.1e-06 | 0.3071398 |
| 0.51 | -0.5201700 | 0.1147119 | -0.7450012 | -0.2953388 | 5.8e-06 | 0.3071772 |
| 0.52 | -0.5202486 | 0.1136304 | -0.7429601 | -0.2975371 | 4.7e-06 | 0.3072155 |
| 0.53 | -0.5203294 | 0.1125355 | -0.7408948 | -0.2997639 | 3.8e-06 | 0.3072548 |
| 0.54 | -0.5204124 | 0.1114266 | -0.7388045 | -0.3020203 | 3.0e-06 | 0.3072952 |
| 0.55 | -0.5204977 | 0.1103034 | -0.7366884 | -0.3043070 | 2.4e-06 | 0.3073367 |
| 0.56 | -0.5205852 | 0.1091654 | -0.7345455 | -0.3066250 | 1.9e-06 | 0.3073791 |
| 0.57 | -0.5206752 | 0.1080120 | -0.7323749 | -0.3089755 | 1.4e-06 | 0.3074230 |
| 0.58 | -0.5207675 | 0.1068428 | -0.7301755 | -0.3113594 | 1.1e-06 | 0.3074682 |
| 0.59 | -0.5208622 | 0.1056572 | -0.7279464 | -0.3137779 | 8.0e-07 | 0.3075148 |
| 0.60 | -0.5209594 | 0.1044545 | -0.7256864 | -0.3162323 | 6.0e-07 | 0.3075629 |
| 0.61 | -0.5210590 | 0.1032341 | -0.7233942 | -0.3187239 | 4.0e-07 | 0.3076126 |
| 0.62 | -0.5211613 | 0.1019954 | -0.7210686 | -0.3212539 | 3.0e-07 | 0.3076639 |
| 0.63 | -0.5212661 | 0.1007377 | -0.7187083 | -0.3238239 | 2.0e-07 | 0.3077170 |
| 0.64 | -0.5213736 | 0.0994601 | -0.7163117 | -0.3264354 | 2.0e-07 | 0.3077721 |
| 0.65 | -0.5214837 | 0.0981618 | -0.7138774 | -0.3290900 | 1.0e-07 | 0.3078293 |
| 0.66 | -0.5215966 | 0.0968421 | -0.7114036 | -0.3317896 | 1.0e-07 | 0.3078888 |
| 0.67 | -0.5217123 | 0.0954998 | -0.7088885 | -0.3345361 | 0.0e+00 | 0.3079508 |
| 0.68 | -0.5218308 | 0.0941341 | -0.7063303 | -0.3373314 | 0.0e+00 | 0.3080156 |
| 0.69 | -0.5219523 | 0.0927438 | -0.7037267 | -0.3401778 | 0.0e+00 | 0.3080835 |
| 0.70 | -0.5220767 | 0.0913277 | -0.7010757 | -0.3430776 | 0.0e+00 | 0.3081548 |
| 0.71 | -0.5222041 | 0.0898847 | -0.6983748 | -0.3460334 | 0.0e+00 | 0.3082299 |
| 0.72 | -0.5223346 | 0.0884132 | -0.6956212 | -0.3490479 | 0.0e+00 | 0.3083094 |
| 0.73 | -0.5224682 | 0.0869118 | -0.6928122 | -0.3521242 | 0.0e+00 | 0.3083939 |
| 0.74 | -0.5226050 | 0.0853789 | -0.6899445 | -0.3552656 | 0.0e+00 | 0.3084841 |
| 0.75 | -0.5227452 | 0.0838126 | -0.6870148 | -0.3584755 | 0.0e+00 | 0.3085811 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.5227 Min. :0.08381 Min. :-0.7470
1st Qu.:0.5625 1st Qu.:-0.5219 1st Qu.:0.09309 1st Qu.:-0.7340
Median :0.6250 Median :-0.5212 Median :0.10137 Median :-0.7199
Mean :0.6250 Mean :-0.5213 Mean :0.10081 Mean :-0.7189
3rd Qu.:0.6875 3rd Qu.:-0.5206 3rd Qu.:0.10888 3rd Qu.:-0.7044
Max. :0.7500 Max. :-0.5201 Max. :0.11578 Max. :-0.6870
ci.ub pvalue sigma2
Min. :-0.3585 Min. :4.460e-10 Min. :0.3071
1st Qu.:-0.3395 1st Qu.:2.109e-08 1st Qu.:0.3074
Median :-0.3225 Median :2.756e-07 Median :0.3077
Mean :-0.3237 Mean :1.305e-06 Mean :0.3077
3rd Qu.:-0.3072 3rd Qu.:1.748e-06 3rd Qu.:0.3081
Max. :-0.2932 Max. :7.053e-06 Max. :0.3086
Higher level only
Code
## HIGHER LEVEL ONLY ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$order_hi <- as.factor(ifelse(dataN14_control$FB_order == 2, 1, 0))
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = order_hi)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.4963895 | 0.1179525 | -0.7275722 | -0.2652069 | 2.57e-05 | 0.3901608 |
| 0.51 | -0.4969280 | 0.1168266 | -0.7259040 | -0.2679520 | 2.10e-05 | 0.3904365 |
| 0.52 | -0.4974700 | 0.1156879 | -0.7242141 | -0.2707260 | 1.71e-05 | 0.3907100 |
| 0.53 | -0.4980156 | 0.1145358 | -0.7225016 | -0.2735295 | 1.37e-05 | 0.3909811 |
| 0.54 | -0.4985646 | 0.1133700 | -0.7207657 | -0.2763635 | 1.09e-05 | 0.3912493 |
| 0.55 | -0.4991172 | 0.1121900 | -0.7190056 | -0.2792288 | 8.60e-06 | 0.3915143 |
| 0.56 | -0.4996734 | 0.1109954 | -0.7172204 | -0.2821263 | 6.70e-06 | 0.3917759 |
| 0.57 | -0.5002331 | 0.1097857 | -0.7154092 | -0.2850571 | 5.20e-06 | 0.3920335 |
| 0.58 | -0.5007964 | 0.1085603 | -0.7135708 | -0.2880221 | 4.00e-06 | 0.3922868 |
| 0.59 | -0.5013633 | 0.1073188 | -0.7117042 | -0.2910224 | 3.00e-06 | 0.3925352 |
| 0.60 | -0.5019338 | 0.1060604 | -0.7098084 | -0.2940592 | 2.20e-06 | 0.3927784 |
| 0.61 | -0.5025078 | 0.1047847 | -0.7078820 | -0.2971337 | 1.60e-06 | 0.3930157 |
| 0.62 | -0.5030855 | 0.1034908 | -0.7059238 | -0.3002472 | 1.20e-06 | 0.3932465 |
| 0.63 | -0.5036668 | 0.1021782 | -0.7039325 | -0.3034011 | 8.00e-07 | 0.3934702 |
| 0.64 | -0.5042517 | 0.1008461 | -0.7019065 | -0.3065969 | 6.00e-07 | 0.3936861 |
| 0.65 | -0.5048402 | 0.0994937 | -0.6998443 | -0.3098361 | 4.00e-07 | 0.3938935 |
| 0.66 | -0.5054323 | 0.0981201 | -0.6977442 | -0.3131205 | 3.00e-07 | 0.3940914 |
| 0.67 | -0.5060281 | 0.0967244 | -0.6956044 | -0.3164519 | 2.00e-07 | 0.3942790 |
| 0.68 | -0.5066275 | 0.0953056 | -0.6934230 | -0.3198321 | 1.00e-07 | 0.3944551 |
| 0.69 | -0.5072306 | 0.0938626 | -0.6911979 | -0.3232633 | 1.00e-07 | 0.3946188 |
| 0.70 | -0.5078373 | 0.0923943 | -0.6889268 | -0.3267478 | 0.00e+00 | 0.3947688 |
| 0.71 | -0.5084477 | 0.0908995 | -0.6866074 | -0.3302880 | 0.00e+00 | 0.3949036 |
| 0.72 | -0.5090617 | 0.0893767 | -0.6842369 | -0.3338865 | 0.00e+00 | 0.3950217 |
| 0.73 | -0.5096794 | 0.0878246 | -0.6818124 | -0.3375463 | 0.00e+00 | 0.3951219 |
| 0.74 | -0.5103007 | 0.0862415 | -0.6793309 | -0.3412705 | 0.00e+00 | 0.3952014 |
| 0.75 | -0.5109257 | 0.0846256 | -0.6767889 | -0.3450625 | 0.00e+00 | 0.3952585 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.5109 Min. :0.08463 Min. :-0.7276
1st Qu.:0.5625 1st Qu.:-0.5071 1st Qu.:0.09422 1st Qu.:-0.7168
Median :0.6250 Median :-0.5034 Median :0.10283 Median :-0.7049
Mean :0.6250 Mean :-0.5035 Mean :0.10229 Mean :-0.7040
3rd Qu.:0.6875 3rd Qu.:-0.4998 3rd Qu.:0.11069 3rd Qu.:-0.6918
Max. :0.7500 Max. :-0.4964 Max. :0.11795 Max. :-0.6768
ci.ub pvalue sigma2
Min. :-0.3451 Min. :1.565e-09 Min. :0.3902
1st Qu.:-0.3224 1st Qu.:7.543e-08 1st Qu.:0.3918
Median :-0.3018 Median :9.961e-07 Median :0.3934
Mean :-0.3030 Mean :4.750e-06 Mean :0.3931
3rd Qu.:-0.2829 3rd Qu.:6.355e-06 3rd Qu.:0.3946
Max. :-0.2652 Max. :2.572e-05 Max. :0.3953
Both lower and higher level
Code
## BOTH LOWER AND HIGHER LEVEL ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$order_hilow <- ifelse(dataN14_control$FB_order == 3, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = order_hilow)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.3428719 | 0.5027374 | -1.328219 | 0.6424754 | 0.4952327 | 0.3837353 |
| 0.51 | -0.3422861 | 0.5020673 | -1.326320 | 0.6417477 | 0.4953949 | 0.3835347 |
| 0.52 | -0.3416840 | 0.5013923 | -1.324395 | 0.6410269 | 0.4955739 | 0.3833278 |
| 0.53 | -0.3410648 | 0.5007128 | -1.322444 | 0.6403143 | 0.4957712 | 0.3831151 |
| 0.54 | -0.3404276 | 0.5000283 | -1.320465 | 0.6396099 | 0.4959876 | 0.3828958 |
| 0.55 | -0.3397712 | 0.4993385 | -1.318457 | 0.6389143 | 0.4962243 | 0.3826694 |
| 0.56 | -0.3390946 | 0.4986432 | -1.316417 | 0.6382281 | 0.4964826 | 0.3824356 |
| 0.57 | -0.3383965 | 0.4979420 | -1.314345 | 0.6375518 | 0.4967639 | 0.3821938 |
| 0.58 | -0.3376757 | 0.4972345 | -1.312237 | 0.6368861 | 0.4970698 | 0.3819435 |
| 0.59 | -0.3369306 | 0.4965205 | -1.310093 | 0.6362317 | 0.4974018 | 0.3816841 |
| 0.60 | -0.3361598 | 0.4957996 | -1.307909 | 0.6355895 | 0.4977618 | 0.3814149 |
| 0.61 | -0.3353615 | 0.4950712 | -1.305683 | 0.6349602 | 0.4981518 | 0.3811352 |
| 0.62 | -0.3345338 | 0.4943349 | -1.303413 | 0.6343448 | 0.4985740 | 0.3808442 |
| 0.63 | -0.3336747 | 0.4935903 | -1.301094 | 0.6337445 | 0.4990308 | 0.3805412 |
| 0.64 | -0.3327819 | 0.4928366 | -1.298724 | 0.6331602 | 0.4995248 | 0.3802251 |
| 0.65 | -0.3318528 | 0.4920733 | -1.296299 | 0.6325932 | 0.5000589 | 0.3798949 |
| 0.66 | -0.3308847 | 0.4912997 | -1.293815 | 0.6320451 | 0.5006366 | 0.3795495 |
| 0.67 | -0.3298745 | 0.4905150 | -1.291266 | 0.6315172 | 0.5012613 | 0.3791875 |
| 0.68 | -0.3288188 | 0.4897183 | -1.288649 | 0.6310115 | 0.5019372 | 0.3788077 |
| 0.69 | -0.3277136 | 0.4889086 | -1.285957 | 0.6305298 | 0.5026690 | 0.3784083 |
| 0.70 | -0.3265545 | 0.4880849 | -1.283183 | 0.6300744 | 0.5034619 | 0.3779876 |
| 0.71 | -0.3253367 | 0.4872459 | -1.280321 | 0.6296478 | 0.5043217 | 0.3775436 |
| 0.72 | -0.3240545 | 0.4863903 | -1.277362 | 0.6292530 | 0.5052554 | 0.3770741 |
| 0.73 | -0.3227014 | 0.4855165 | -1.274296 | 0.6288934 | 0.5062706 | 0.3765765 |
| 0.74 | -0.3212702 | 0.4846227 | -1.271113 | 0.6285729 | 0.5073764 | 0.3760481 |
| 0.75 | -0.3197524 | 0.4837074 | -1.267801 | 0.6282966 | 0.5085834 | 0.3754861 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.3429 Min. :0.4837 Min. :-1.328
1st Qu.:0.5625 1st Qu.:-0.3389 1st Qu.:0.4891 1st Qu.:-1.316
Median :0.6250 Median :-0.3341 Median :0.4940 Median :-1.302
Mean :0.6250 Mean :-0.3331 Mean :0.4937 Mean :-1.301
3rd Qu.:0.6875 3rd Qu.:-0.3280 3rd Qu.:0.4985 3rd Qu.:-1.287
Max. :0.7500 Max. :-0.3198 Max. :0.5027 Max. :-1.268
ci.ub pvalue sigma2
Min. :0.6283 Min. :0.4952 Min. :0.3755
1st Qu.:0.6307 1st Qu.:0.4966 1st Qu.:0.3785
Median :0.6340 Median :0.4988 Median :0.3807
Mean :0.6345 Mean :0.4999 Mean :0.3803
3rd Qu.:0.6381 3rd Qu.:0.5025 3rd Qu.:0.3824
Max. :0.6425 Max. :0.5086 Max. :0.3837
Specificity
Code
## SPECIFICITY ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$spec <- as.factor(dataN14_control$FB_specificity)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = spec)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.3428719 | 0.5027374 | -0.6424754 | 1.328219 | 0.4952327 | 0.3837353 |
| 0.51 | 0.3422861 | 0.5020673 | -0.6417477 | 1.326320 | 0.4953949 | 0.3835347 |
| 0.52 | 0.3416840 | 0.5013923 | -0.6410269 | 1.324395 | 0.4955739 | 0.3833278 |
| 0.53 | 0.3410648 | 0.5007128 | -0.6403143 | 1.322444 | 0.4957712 | 0.3831151 |
| 0.54 | 0.3404276 | 0.5000283 | -0.6396099 | 1.320465 | 0.4959876 | 0.3828958 |
| 0.55 | 0.3397712 | 0.4993385 | -0.6389143 | 1.318457 | 0.4962243 | 0.3826694 |
| 0.56 | 0.3390946 | 0.4986432 | -0.6382281 | 1.316417 | 0.4964826 | 0.3824356 |
| 0.57 | 0.3383965 | 0.4979420 | -0.6375518 | 1.314345 | 0.4967639 | 0.3821938 |
| 0.58 | 0.3376757 | 0.4972345 | -0.6368861 | 1.312237 | 0.4970698 | 0.3819435 |
| 0.59 | 0.3369306 | 0.4965205 | -0.6362317 | 1.310093 | 0.4974018 | 0.3816841 |
| 0.60 | 0.3361598 | 0.4957996 | -0.6355895 | 1.307909 | 0.4977618 | 0.3814149 |
| 0.61 | 0.3353615 | 0.4950712 | -0.6349602 | 1.305683 | 0.4981518 | 0.3811352 |
| 0.62 | 0.3345338 | 0.4943349 | -0.6343448 | 1.303413 | 0.4985740 | 0.3808442 |
| 0.63 | 0.3336747 | 0.4935903 | -0.6337445 | 1.301094 | 0.4990308 | 0.3805412 |
| 0.64 | 0.3327819 | 0.4928366 | -0.6331602 | 1.298724 | 0.4995248 | 0.3802251 |
| 0.65 | 0.3318528 | 0.4920733 | -0.6325932 | 1.296299 | 0.5000589 | 0.3798949 |
| 0.66 | 0.3308847 | 0.4912997 | -0.6320451 | 1.293815 | 0.5006366 | 0.3795495 |
| 0.67 | 0.3298745 | 0.4905150 | -0.6315172 | 1.291266 | 0.5012613 | 0.3791875 |
| 0.68 | 0.3288188 | 0.4897183 | -0.6310115 | 1.288649 | 0.5019372 | 0.3788077 |
| 0.69 | 0.3277136 | 0.4889087 | -0.6305298 | 1.285957 | 0.5026690 | 0.3784083 |
| 0.70 | 0.3265545 | 0.4880849 | -0.6300744 | 1.283183 | 0.5034619 | 0.3779876 |
| 0.71 | 0.3253367 | 0.4872459 | -0.6296478 | 1.280321 | 0.5043217 | 0.3775436 |
| 0.72 | 0.3240545 | 0.4863903 | -0.6292530 | 1.277362 | 0.5052554 | 0.3770741 |
| 0.73 | 0.3227014 | 0.4855165 | -0.6288934 | 1.274296 | 0.5062706 | 0.3765765 |
| 0.74 | 0.3212702 | 0.4846227 | -0.6285729 | 1.271113 | 0.5073764 | 0.3760481 |
| 0.75 | 0.3197524 | 0.4837073 | -0.6282966 | 1.267801 | 0.5085834 | 0.3754861 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.3198 Min. :0.4837 Min. :-0.6425
1st Qu.:0.5625 1st Qu.:0.3280 1st Qu.:0.4891 1st Qu.:-0.6381
Median :0.6250 Median :0.3341 Median :0.4940 Median :-0.6340
Mean :0.6250 Mean :0.3331 Mean :0.4937 Mean :-0.6345
3rd Qu.:0.6875 3rd Qu.:0.3389 3rd Qu.:0.4985 3rd Qu.:-0.6307
Max. :0.7500 Max. :0.3429 Max. :0.5027 Max. :-0.6283
ci.ub pvalue sigma2
Min. :1.268 Min. :0.4952 Min. :0.3755
1st Qu.:1.287 1st Qu.:0.4966 1st Qu.:0.3785
Median :1.302 Median :0.4988 Median :0.3807
Mean :1.301 Mean :0.4999 Mean :0.3803
3rd Qu.:1.316 3rd Qu.:0.5025 3rd Qu.:0.3824
Max. :1.328 Max. :0.5086 Max. :0.3837
Tool numbers
Code
## TOOL NUMBERS ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$tool_nr <- as.factor(dataN14_control$FB_tool_numbers)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = tool_nr)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.1335501 | 0.1545890 | -0.4365390 | 0.1694389 | 0.3876407 | 0.3506367 |
| 0.51 | -0.1335603 | 0.1544060 | -0.4361905 | 0.1690700 | 0.3870421 | 0.3501995 |
| 0.52 | -0.1335693 | 0.1542209 | -0.4358366 | 0.1686981 | 0.3864404 | 0.3497538 |
| 0.53 | -0.1335771 | 0.1540335 | -0.4354773 | 0.1683231 | 0.3858353 | 0.3492990 |
| 0.54 | -0.1335836 | 0.1538439 | -0.4351121 | 0.1679448 | 0.3852266 | 0.3488348 |
| 0.55 | -0.1335889 | 0.1536518 | -0.4347409 | 0.1675630 | 0.3846139 | 0.3483604 |
| 0.56 | -0.1335929 | 0.1534571 | -0.4343633 | 0.1671774 | 0.3839969 | 0.3478754 |
| 0.57 | -0.1335955 | 0.1532596 | -0.4339789 | 0.1667878 | 0.3833752 | 0.3473789 |
| 0.58 | -0.1335968 | 0.1530593 | -0.4335875 | 0.1663939 | 0.3827485 | 0.3468704 |
| 0.59 | -0.1335966 | 0.1528559 | -0.4331886 | 0.1659954 | 0.3821162 | 0.3463489 |
| 0.60 | -0.1335950 | 0.1526491 | -0.4327818 | 0.1655918 | 0.3814777 | 0.3458136 |
| 0.61 | -0.1335919 | 0.1524389 | -0.4323666 | 0.1651828 | 0.3808324 | 0.3452635 |
| 0.62 | -0.1335873 | 0.1522249 | -0.4319426 | 0.1647680 | 0.3801795 | 0.3446975 |
| 0.63 | -0.1335812 | 0.1520069 | -0.4315092 | 0.1643467 | 0.3795182 | 0.3441145 |
| 0.64 | -0.1335737 | 0.1517845 | -0.4310659 | 0.1639185 | 0.3788474 | 0.3435132 |
| 0.65 | -0.1335647 | 0.1515575 | -0.4306119 | 0.1634826 | 0.3781661 | 0.3428920 |
| 0.66 | -0.1335542 | 0.1513255 | -0.4301467 | 0.1630383 | 0.3774727 | 0.3422493 |
| 0.67 | -0.1335423 | 0.1510881 | -0.4296695 | 0.1625848 | 0.3767659 | 0.3415834 |
| 0.68 | -0.1335292 | 0.1508447 | -0.4291794 | 0.1621210 | 0.3760436 | 0.3408922 |
| 0.69 | -0.1335148 | 0.1505950 | -0.4286755 | 0.1616459 | 0.3753038 | 0.3401733 |
| 0.70 | -0.1334994 | 0.1503382 | -0.4281569 | 0.1611581 | 0.3745440 | 0.3394242 |
| 0.71 | -0.1334832 | 0.1500738 | -0.4276224 | 0.1606561 | 0.3737611 | 0.3386420 |
| 0.72 | -0.1334664 | 0.1498010 | -0.4270709 | 0.1601381 | 0.3729517 | 0.3378232 |
| 0.73 | -0.1334493 | 0.1495188 | -0.4265009 | 0.1596022 | 0.3721116 | 0.3369642 |
| 0.74 | -0.1334325 | 0.1492264 | -0.4259109 | 0.1590460 | 0.3712357 | 0.3360605 |
| 0.75 | -0.1334165 | 0.1489226 | -0.4252995 | 0.1584665 | 0.3703181 | 0.3351071 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.1336 Min. :0.1489 Min. :-0.4365
1st Qu.:0.5625 1st Qu.:-0.1336 1st Qu.:0.1507 1st Qu.:-0.4343
Median :0.6250 Median :-0.1336 Median :0.1521 Median :-0.4317
Mean :0.6250 Mean :-0.1335 Mean :0.1520 Mean :-0.4314
3rd Qu.:0.6875 3rd Qu.:-0.1335 3rd Qu.:0.1534 3rd Qu.:-0.4288
Max. :0.7500 Max. :-0.1334 Max. :0.1546 Max. :-0.4253
ci.ub pvalue sigma2
Min. :0.1585 Min. :0.3703 Min. :0.3351
1st Qu.:0.1618 1st Qu.:0.3755 1st Qu.:0.3404
Median :0.1646 Median :0.3798 Median :0.3444
Mean :0.1644 Mean :0.3796 Mean :0.3439
3rd Qu.:0.1671 3rd Qu.:0.3838 3rd Qu.:0.3478
Max. :0.1694 Max. :0.3876 Max. :0.3506
Prior knowledge
Code
## PRIOR KNOWLEDGE ############################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$WQP_pre <- as.factor(dataN14_control$WQP_FB_pre)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = WQP_pre)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.1465664 | 0.0307505 | -0.2068362 | -0.0862965 | 1.9e-06 | 0.1548649 |
| 0.51 | -0.1470022 | 0.0306907 | -0.2071549 | -0.0868495 | 1.7e-06 | 0.1571516 |
| 0.52 | -0.1474590 | 0.0306260 | -0.2074848 | -0.0874331 | 1.5e-06 | 0.1594858 |
| 0.53 | -0.1479374 | 0.0305562 | -0.2078264 | -0.0880483 | 1.3e-06 | 0.1618698 |
| 0.54 | -0.1484381 | 0.0304810 | -0.2081798 | -0.0886964 | 1.1e-06 | 0.1643061 |
| 0.55 | -0.1489619 | 0.0304002 | -0.2085453 | -0.0893785 | 1.0e-06 | 0.1667969 |
| 0.56 | -0.1495095 | 0.0303136 | -0.2089231 | -0.0900960 | 8.0e-07 | 0.1693451 |
| 0.57 | -0.1500817 | 0.0302208 | -0.2093133 | -0.0908500 | 7.0e-07 | 0.1719534 |
| 0.58 | -0.1506792 | 0.0301215 | -0.2097163 | -0.0916421 | 6.0e-07 | 0.1746249 |
| 0.59 | -0.1513029 | 0.0300155 | -0.2101321 | -0.0924737 | 5.0e-07 | 0.1773628 |
| 0.60 | -0.1519537 | 0.0299023 | -0.2105610 | -0.0933463 | 4.0e-07 | 0.1801704 |
| 0.61 | -0.1526323 | 0.0297815 | -0.2110031 | -0.0942616 | 3.0e-07 | 0.1830512 |
| 0.62 | -0.1533398 | 0.0296529 | -0.2114584 | -0.0952213 | 2.0e-07 | 0.1860091 |
| 0.63 | -0.1540771 | 0.0295158 | -0.2119270 | -0.0962271 | 2.0e-07 | 0.1890478 |
| 0.64 | -0.1548451 | 0.0293699 | -0.2124091 | -0.0972811 | 1.0e-07 | 0.1921720 |
| 0.65 | -0.1556449 | 0.0292147 | -0.2129046 | -0.0983851 | 1.0e-07 | 0.1953861 |
| 0.66 | -0.1564774 | 0.0290495 | -0.2134134 | -0.0995414 | 1.0e-07 | 0.1986945 |
| 0.67 | -0.1573439 | 0.0288738 | -0.2139356 | -0.1007522 | 1.0e-07 | 0.2021030 |
| 0.68 | -0.1582453 | 0.0286870 | -0.2144708 | -0.1020198 | 0.0e+00 | 0.2056167 |
| 0.69 | -0.1591829 | 0.0284883 | -0.2150189 | -0.1033469 | 0.0e+00 | 0.2092416 |
| 0.70 | -0.1601579 | 0.0282769 | -0.2155797 | -0.1047361 | 0.0e+00 | 0.2129840 |
| 0.71 | -0.1611715 | 0.0280521 | -0.2161526 | -0.1061903 | 0.0e+00 | 0.2168505 |
| 0.72 | -0.1622250 | 0.0278129 | -0.2167373 | -0.1077126 | 0.0e+00 | 0.2208484 |
| 0.73 | -0.1633198 | 0.0275583 | -0.2173331 | -0.1093064 | 0.0e+00 | 0.2249858 |
| 0.74 | -0.1644573 | 0.0272873 | -0.2179393 | -0.1109752 | 0.0e+00 | 0.2292706 |
| 0.75 | -0.1656389 | 0.0269985 | -0.2185549 | -0.1127229 | 0.0e+00 | 0.2337124 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.1656 Min. :0.02700 Min. :-0.2186
1st Qu.:0.5625 1st Qu.:-0.1589 1st Qu.:0.02854 1st Qu.:-0.2149
Median :0.6250 Median :-0.1537 Median :0.02958 Median :-0.2117
Mean :0.6250 Mean :-0.1546 Mean :0.02933 Mean :-0.2121
3rd Qu.:0.6875 3rd Qu.:-0.1497 3rd Qu.:0.03029 3rd Qu.:-0.2090
Max. :0.7500 Max. :-0.1466 Max. :0.03075 Max. :-0.2068
ci.ub pvalue sigma2
Min. :-0.11272 Min. :8.509e-10 Min. :0.1549
1st Qu.:-0.10302 1st Qu.:2.592e-08 1st Qu.:0.1700
Median :-0.09572 Median :2.057e-07 Median :0.1875
Mean :-0.09707 Mean :4.785e-07 Mean :0.1899
3rd Qu.:-0.09028 3rd Qu.:7.809e-07 3rd Qu.:0.2083
Max. :-0.08630 Max. :1.876e-06 Max. :0.2337
Setting
Code
## SETTING ######################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$setting <- as.factor(dataN14_control$setting)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = setting)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.4974945 | 0.4469210 | -0.3784546 | 1.373444 | 0.2656398 | 0.3205693 |
| 0.51 | 0.4965195 | 0.4465553 | -0.3787129 | 1.371752 | 0.2661863 | 0.3207612 |
| 0.52 | 0.4955467 | 0.4461830 | -0.3789558 | 1.370049 | 0.2667252 | 0.3209444 |
| 0.53 | 0.4945763 | 0.4458036 | -0.3791828 | 1.368335 | 0.2672560 | 0.3211184 |
| 0.54 | 0.4936083 | 0.4454170 | -0.3793929 | 1.366610 | 0.2677782 | 0.3212827 |
| 0.55 | 0.4926427 | 0.4450225 | -0.3795854 | 1.364871 | 0.2682912 | 0.3214366 |
| 0.56 | 0.4916797 | 0.4446200 | -0.3797594 | 1.363119 | 0.2687947 | 0.3215796 |
| 0.57 | 0.4907193 | 0.4442088 | -0.3799140 | 1.361352 | 0.2692878 | 0.3217108 |
| 0.58 | 0.4897615 | 0.4437885 | -0.3800480 | 1.359571 | 0.2697701 | 0.3218296 |
| 0.59 | 0.4888065 | 0.4433586 | -0.3801604 | 1.357773 | 0.2702408 | 0.3219351 |
| 0.60 | 0.4878542 | 0.4429184 | -0.3802500 | 1.355958 | 0.2706992 | 0.3220264 |
| 0.61 | 0.4869047 | 0.4424674 | -0.3803155 | 1.354125 | 0.2711445 | 0.3221026 |
| 0.62 | 0.4859580 | 0.4420048 | -0.3803556 | 1.352272 | 0.2715760 | 0.3221626 |
| 0.63 | 0.4850142 | 0.4415300 | -0.3803686 | 1.350397 | 0.2719926 | 0.3222053 |
| 0.64 | 0.4840732 | 0.4410419 | -0.3803530 | 1.348499 | 0.2723934 | 0.3222292 |
| 0.65 | 0.4831351 | 0.4405398 | -0.3803070 | 1.346577 | 0.2727774 | 0.3222332 |
| 0.66 | 0.4821998 | 0.4400225 | -0.3802286 | 1.344628 | 0.2731434 | 0.3222155 |
| 0.67 | 0.4812672 | 0.4394891 | -0.3801156 | 1.342650 | 0.2734901 | 0.3221745 |
| 0.68 | 0.4803373 | 0.4389382 | -0.3799657 | 1.340640 | 0.2738162 | 0.3221082 |
| 0.69 | 0.4794099 | 0.4383684 | -0.3797763 | 1.338596 | 0.2741202 | 0.3220147 |
| 0.70 | 0.4784849 | 0.4377782 | -0.3795446 | 1.336514 | 0.2744005 | 0.3218915 |
| 0.71 | 0.4775621 | 0.4371659 | -0.3792672 | 1.334391 | 0.2746552 | 0.3217360 |
| 0.72 | 0.4766412 | 0.4365294 | -0.3789407 | 1.332223 | 0.2748824 | 0.3215452 |
| 0.73 | 0.4757219 | 0.4358667 | -0.3785611 | 1.330005 | 0.2750797 | 0.3213158 |
| 0.74 | 0.4748039 | 0.4351752 | -0.3781237 | 1.327732 | 0.2752448 | 0.3210441 |
| 0.75 | 0.4738866 | 0.4344520 | -0.3776236 | 1.325397 | 0.2753746 | 0.3207257 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.4739 Min. :0.4345 Min. :-0.3804
1st Qu.:0.5625 1st Qu.:0.4796 1st Qu.:0.4385 1st Qu.:-0.3802
Median :0.6250 Median :0.4855 Median :0.4418 Median :-0.3798
Mean :0.6250 Mean :0.4856 Mean :0.4414 Mean :-0.3795
3rd Qu.:0.6875 3rd Qu.:0.4914 3rd Qu.:0.4445 3rd Qu.:-0.3790
Max. :0.7500 Max. :0.4975 Max. :0.4469 Max. :-0.3776
ci.ub pvalue sigma2
Min. :1.325 Min. :0.2656 Min. :0.3206
1st Qu.:1.339 1st Qu.:0.2689 1st Qu.:0.3213
Median :1.351 Median :0.2718 Median :0.3218
Mean :1.351 Mean :0.2713 Mean :0.3216
3rd Qu.:1.363 3rd Qu.:0.2740 3rd Qu.:0.3221
Max. :1.373 Max. :0.2754 Max. :0.3222
Educational level
Code
## EDUCATIONAL LEVEL dummy-coded #####################
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$edu <- as.factor(dataN14_control$education_dummy)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = edu)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.4974945 | 0.4469210 | -1.373444 | 0.3784546 | 0.2656398 | 0.3205693 |
| 0.51 | -0.4965195 | 0.4465553 | -1.371752 | 0.3787129 | 0.2661863 | 0.3207612 |
| 0.52 | -0.4955467 | 0.4461830 | -1.370049 | 0.3789558 | 0.2667252 | 0.3209444 |
| 0.53 | -0.4945763 | 0.4458036 | -1.368335 | 0.3791828 | 0.2672560 | 0.3211184 |
| 0.54 | -0.4936083 | 0.4454170 | -1.366610 | 0.3793929 | 0.2677782 | 0.3212827 |
| 0.55 | -0.4926427 | 0.4450225 | -1.364871 | 0.3795854 | 0.2682912 | 0.3214366 |
| 0.56 | -0.4916797 | 0.4446200 | -1.363119 | 0.3797594 | 0.2687947 | 0.3215796 |
| 0.57 | -0.4907193 | 0.4442088 | -1.361352 | 0.3799139 | 0.2692878 | 0.3217108 |
| 0.58 | -0.4897615 | 0.4437885 | -1.359571 | 0.3800480 | 0.2697701 | 0.3218296 |
| 0.59 | -0.4888065 | 0.4433586 | -1.357773 | 0.3801604 | 0.2702408 | 0.3219351 |
| 0.60 | -0.4878542 | 0.4429184 | -1.355958 | 0.3802500 | 0.2706992 | 0.3220264 |
| 0.61 | -0.4869047 | 0.4424674 | -1.354125 | 0.3803155 | 0.2711445 | 0.3221026 |
| 0.62 | -0.4859580 | 0.4420048 | -1.352272 | 0.3803556 | 0.2715760 | 0.3221626 |
| 0.63 | -0.4850142 | 0.4415300 | -1.350397 | 0.3803686 | 0.2719926 | 0.3222053 |
| 0.64 | -0.4840732 | 0.4410419 | -1.348499 | 0.3803530 | 0.2723934 | 0.3222292 |
| 0.65 | -0.4831351 | 0.4405398 | -1.346577 | 0.3803070 | 0.2727774 | 0.3222332 |
| 0.66 | -0.4821998 | 0.4400225 | -1.344628 | 0.3802286 | 0.2731434 | 0.3222155 |
| 0.67 | -0.4812672 | 0.4394891 | -1.342650 | 0.3801156 | 0.2734901 | 0.3221745 |
| 0.68 | -0.4803373 | 0.4389382 | -1.340640 | 0.3799657 | 0.2738162 | 0.3221082 |
| 0.69 | -0.4794099 | 0.4383684 | -1.338596 | 0.3797763 | 0.2741202 | 0.3220147 |
| 0.70 | -0.4784849 | 0.4377782 | -1.336514 | 0.3795446 | 0.2744005 | 0.3218915 |
| 0.71 | -0.4775621 | 0.4371659 | -1.334391 | 0.3792672 | 0.2746552 | 0.3217360 |
| 0.72 | -0.4766412 | 0.4365294 | -1.332223 | 0.3789407 | 0.2748824 | 0.3215452 |
| 0.73 | -0.4757219 | 0.4358667 | -1.330005 | 0.3785611 | 0.2750797 | 0.3213158 |
| 0.74 | -0.4748039 | 0.4351752 | -1.327732 | 0.3781237 | 0.2752448 | 0.3210441 |
| 0.75 | -0.4738866 | 0.4344520 | -1.325397 | 0.3776236 | 0.2753746 | 0.3207257 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.4975 Min. :0.4345 Min. :-1.373
1st Qu.:0.5625 1st Qu.:-0.4914 1st Qu.:0.4385 1st Qu.:-1.363
Median :0.6250 Median :-0.4855 Median :0.4418 Median :-1.351
Mean :0.6250 Mean :-0.4856 Mean :0.4414 Mean :-1.351
3rd Qu.:0.6875 3rd Qu.:-0.4796 3rd Qu.:0.4445 3rd Qu.:-1.339
Max. :0.7500 Max. :-0.4739 Max. :0.4469 Max. :-1.325
ci.ub pvalue sigma2
Min. :0.3776 Min. :0.2656 Min. :0.3206
1st Qu.:0.3790 1st Qu.:0.2689 1st Qu.:0.3213
Median :0.3798 Median :0.2718 Median :0.3218
Mean :0.3795 Mean :0.2713 Mean :0.3216
3rd Qu.:0.3802 3rd Qu.:0.2740 3rd Qu.:0.3221
Max. :0.3804 Max. :0.2754 Max. :0.3222
Post measure
Code
##POST MEASURE##############
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$post_measure <- as.factor(dataN14_control$post_measure)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = post_measure)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.4085439 | 0.4784760 | -1.346340 | 0.5292518 | 0.3931914 | 0.3661479 |
| 0.51 | -0.4081858 | 0.4778078 | -1.344672 | 0.5283002 | 0.3929450 | 0.3658311 |
| 0.52 | -0.4078150 | 0.4771346 | -1.342982 | 0.5273516 | 0.3927079 | 0.3655085 |
| 0.53 | -0.4074310 | 0.4764562 | -1.341268 | 0.5264060 | 0.3924803 | 0.3651795 |
| 0.54 | -0.4070331 | 0.4757723 | -1.339530 | 0.5254635 | 0.3922630 | 0.3648438 |
| 0.55 | -0.4066206 | 0.4750827 | -1.337765 | 0.5245243 | 0.3920562 | 0.3645009 |
| 0.56 | -0.4061929 | 0.4743870 | -1.335974 | 0.5235885 | 0.3918606 | 0.3641505 |
| 0.57 | -0.4057493 | 0.4736849 | -1.334155 | 0.5226560 | 0.3916767 | 0.3637919 |
| 0.58 | -0.4052889 | 0.4729760 | -1.332305 | 0.5217270 | 0.3915051 | 0.3634247 |
| 0.59 | -0.4048109 | 0.4722600 | -1.330423 | 0.5208016 | 0.3913464 | 0.3630481 |
| 0.60 | -0.4043145 | 0.4715363 | -1.328509 | 0.5198797 | 0.3912013 | 0.3626616 |
| 0.61 | -0.4037985 | 0.4708045 | -1.326558 | 0.5189614 | 0.3910705 | 0.3622644 |
| 0.62 | -0.4032621 | 0.4700641 | -1.324571 | 0.5180467 | 0.3909546 | 0.3618556 |
| 0.63 | -0.4027040 | 0.4693145 | -1.322544 | 0.5171355 | 0.3908546 | 0.3614344 |
| 0.64 | -0.4021231 | 0.4685550 | -1.320474 | 0.5162278 | 0.3907711 | 0.3609998 |
| 0.65 | -0.4015181 | 0.4677849 | -1.318360 | 0.5153234 | 0.3907052 | 0.3605506 |
| 0.66 | -0.4008876 | 0.4670033 | -1.316197 | 0.5144221 | 0.3906575 | 0.3600856 |
| 0.67 | -0.4002300 | 0.4662095 | -1.313984 | 0.5135239 | 0.3906293 | 0.3596034 |
| 0.68 | -0.3995437 | 0.4654024 | -1.311716 | 0.5126282 | 0.3906215 | 0.3591024 |
| 0.69 | -0.3988270 | 0.4645811 | -1.309389 | 0.5117352 | 0.3906353 | 0.3585815 |
| 0.70 | -0.3980777 | 0.4637438 | -1.306999 | 0.5108434 | 0.3906715 | 0.3580377 |
| 0.71 | -0.3972939 | 0.4628893 | -1.304540 | 0.5099525 | 0.3907316 | 0.3574691 |
| 0.72 | -0.3964731 | 0.4620161 | -1.302008 | 0.5090618 | 0.3908168 | 0.3568734 |
| 0.73 | -0.3956128 | 0.4611223 | -1.299396 | 0.5081703 | 0.3909285 | 0.3562477 |
| 0.74 | -0.3947102 | 0.4602059 | -1.296697 | 0.5072767 | 0.3910682 | 0.3555888 |
| 0.75 | -0.3937619 | 0.4592643 | -1.293903 | 0.5063795 | 0.3912374 | 0.3548930 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.4085 Min. :0.4593 Min. :-1.346
1st Qu.:0.5625 1st Qu.:-0.4061 1st Qu.:0.4648 1st Qu.:-1.336
Median :0.6250 Median :-0.4030 Median :0.4697 Median :-1.324
Mean :0.6250 Mean :-0.4023 Mean :0.4694 Mean :-1.322
3rd Qu.:0.6875 3rd Qu.:-0.3990 3rd Qu.:0.4742 3rd Qu.:-1.310
Max. :0.7500 Max. :-0.3938 Max. :0.4785 Max. :-1.294
ci.ub pvalue sigma2
Min. :0.5064 Min. :0.3906 Min. :0.3549
1st Qu.:0.5120 1st Qu.:0.3907 1st Qu.:0.3587
Median :0.5176 Median :0.3911 Median :0.3616
Mean :0.5177 Mean :0.3914 Mean :0.3613
3rd Qu.:0.5234 3rd Qu.:0.3918 3rd Qu.:0.3641
Max. :0.5293 Max. :0.3932 Max. :0.3661
Amount
Code
## AMOUNT ######
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$amount <- as.factor(dataN14_control$amount)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = amount)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.1604791 | 0.4558302 | -1.053890 | 0.7329316 | 0.7247940 | 0.3767783 |
| 0.51 | -0.1602618 | 0.4552061 | -1.052449 | 0.7319257 | 0.7247900 | 0.3764834 |
| 0.52 | -0.1600369 | 0.4545774 | -1.050992 | 0.7309185 | 0.7247959 | 0.3761818 |
| 0.53 | -0.1598042 | 0.4539441 | -1.049518 | 0.7299099 | 0.7248121 | 0.3758730 |
| 0.54 | -0.1595632 | 0.4533058 | -1.048026 | 0.7288999 | 0.7248390 | 0.3755566 |
| 0.55 | -0.1593137 | 0.4526623 | -1.046516 | 0.7278882 | 0.7248771 | 0.3752320 |
| 0.56 | -0.1590553 | 0.4520134 | -1.044985 | 0.7268747 | 0.7249269 | 0.3748989 |
| 0.57 | -0.1587876 | 0.4513587 | -1.043434 | 0.7258591 | 0.7249889 | 0.3745565 |
| 0.58 | -0.1585102 | 0.4506979 | -1.041862 | 0.7248414 | 0.7250636 | 0.3742044 |
| 0.59 | -0.1582228 | 0.4500306 | -1.040267 | 0.7238211 | 0.7251516 | 0.3738417 |
| 0.60 | -0.1579248 | 0.4493566 | -1.038647 | 0.7227979 | 0.7252534 | 0.3734678 |
| 0.61 | -0.1576158 | 0.4486753 | -1.037003 | 0.7217716 | 0.7253698 | 0.3730819 |
| 0.62 | -0.1572952 | 0.4479862 | -1.035332 | 0.7207417 | 0.7255012 | 0.3726830 |
| 0.63 | -0.1569627 | 0.4472890 | -1.033633 | 0.7197077 | 0.7256483 | 0.3722701 |
| 0.64 | -0.1566175 | 0.4465830 | -1.031904 | 0.7186690 | 0.7258119 | 0.3718422 |
| 0.65 | -0.1562592 | 0.4458676 | -1.030144 | 0.7176251 | 0.7259927 | 0.3713980 |
| 0.66 | -0.1558871 | 0.4451420 | -1.028350 | 0.7165752 | 0.7261913 | 0.3709363 |
| 0.67 | -0.1555006 | 0.4444056 | -1.026520 | 0.7155184 | 0.7264086 | 0.3704554 |
| 0.68 | -0.1550989 | 0.4436575 | -1.024652 | 0.7144538 | 0.7266453 | 0.3699537 |
| 0.69 | -0.1546813 | 0.4428966 | -1.022743 | 0.7133802 | 0.7269022 | 0.3694293 |
| 0.70 | -0.1542470 | 0.4421220 | -1.020790 | 0.7122962 | 0.7271802 | 0.3688802 |
| 0.71 | -0.1537953 | 0.4413324 | -1.018791 | 0.7112004 | 0.7274801 | 0.3683040 |
| 0.72 | -0.1533252 | 0.4405264 | -1.016741 | 0.7100908 | 0.7278027 | 0.3676980 |
| 0.73 | -0.1528357 | 0.4397025 | -1.014637 | 0.7089654 | 0.7281490 | 0.3670592 |
| 0.74 | -0.1523259 | 0.4388589 | -1.012474 | 0.7078218 | 0.7285197 | 0.3663841 |
| 0.75 | -0.1517946 | 0.4379935 | -1.010246 | 0.7066570 | 0.7289158 | 0.3656688 |
Code
summary(sensitivity_moderator) #mean = -.157, p = .726 ri_t beta se ci.lb
Min. :0.5000 Min. :-0.1605 Min. :0.4380 Min. :-1.054
1st Qu.:0.5625 1st Qu.:-0.1590 1st Qu.:0.4431 1st Qu.:-1.045
Median :0.6250 Median :-0.1571 Median :0.4476 Median :-1.034
Mean :0.6250 Mean :-0.1568 Mean :0.4474 Mean :-1.034
3rd Qu.:0.6875 3rd Qu.:-0.1548 3rd Qu.:0.4518 3rd Qu.:-1.023
Max. :0.7500 Max. :-0.1518 Max. :0.4558 Max. :-1.010
ci.ub pvalue sigma2
Min. :0.7067 Min. :0.7248 Min. :0.3657
1st Qu.:0.7136 1st Qu.:0.7249 1st Qu.:0.3696
Median :0.7202 Median :0.7256 Median :0.3725
Mean :0.7201 Mean :0.7260 Mean :0.3720
3rd Qu.:0.7266 3rd Qu.:0.7268 3rd Qu.:0.3748
Max. :0.7329 Max. :0.7289 Max. :0.3768
Coding
Code
## CODING #######
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$coding <- as.factor(dataN14_control$coding)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = coding)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.4845553 | 0.2919013 | -1.056671 | 0.0875608 | 0.0969151 | 0.2628769 |
| 0.51 | -0.4841750 | 0.2915151 | -1.055534 | 0.0871841 | 0.0967352 | 0.2626270 |
| 0.52 | -0.4837856 | 0.2911258 | -1.054382 | 0.0868105 | 0.0965579 | 0.2623731 |
| 0.53 | -0.4833867 | 0.2907333 | -1.053214 | 0.0864401 | 0.0963832 | 0.2621148 |
| 0.54 | -0.4829780 | 0.2903375 | -1.052029 | 0.0860730 | 0.0962113 | 0.2618520 |
| 0.55 | -0.4825588 | 0.2899381 | -1.050827 | 0.0857094 | 0.0960423 | 0.2615844 |
| 0.56 | -0.4821288 | 0.2895350 | -1.049607 | 0.0853494 | 0.0958762 | 0.2613115 |
| 0.57 | -0.4816873 | 0.2891279 | -1.048368 | 0.0849930 | 0.0957133 | 0.2610330 |
| 0.58 | -0.4812339 | 0.2887167 | -1.047108 | 0.0846404 | 0.0955536 | 0.2607485 |
| 0.59 | -0.4807678 | 0.2883010 | -1.045827 | 0.0842918 | 0.0953972 | 0.2604576 |
| 0.60 | -0.4802885 | 0.2878806 | -1.044524 | 0.0839471 | 0.0952443 | 0.2601597 |
| 0.61 | -0.4797952 | 0.2874551 | -1.043197 | 0.0836065 | 0.0950949 | 0.2598543 |
| 0.62 | -0.4792871 | 0.2870243 | -1.041844 | 0.0832701 | 0.0949493 | 0.2595407 |
| 0.63 | -0.4787635 | 0.2865876 | -1.040465 | 0.0829378 | 0.0948074 | 0.2592183 |
| 0.64 | -0.4782235 | 0.2861447 | -1.039057 | 0.0826098 | 0.0946694 | 0.2588863 |
| 0.65 | -0.4776661 | 0.2856951 | -1.037618 | 0.0822860 | 0.0945355 | 0.2585437 |
| 0.66 | -0.4770902 | 0.2852382 | -1.036147 | 0.0819664 | 0.0944055 | 0.2581898 |
| 0.67 | -0.4764947 | 0.2847734 | -1.034640 | 0.0816509 | 0.0942798 | 0.2578232 |
| 0.68 | -0.4758785 | 0.2843001 | -1.033096 | 0.0813394 | 0.0941581 | 0.2574429 |
| 0.69 | -0.4752402 | 0.2838173 | -1.031512 | 0.0810316 | 0.0940407 | 0.2570473 |
| 0.70 | -0.4745783 | 0.2833244 | -1.029884 | 0.0807273 | 0.0939273 | 0.2566350 |
| 0.71 | -0.4738912 | 0.2828201 | -1.028208 | 0.0804260 | 0.0938181 | 0.2562040 |
| 0.72 | -0.4731772 | 0.2823034 | -1.026482 | 0.0801274 | 0.0937128 | 0.2557523 |
| 0.73 | -0.4724344 | 0.2817730 | -1.024699 | 0.0798306 | 0.0936112 | 0.2552776 |
| 0.74 | -0.4716606 | 0.2812273 | -1.022856 | 0.0795349 | 0.0935131 | 0.2547771 |
| 0.75 | -0.4708534 | 0.2806646 | -1.020946 | 0.0792392 | 0.0934180 | 0.2542477 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.4846 Min. :0.2807 Min. :-1.057
1st Qu.:0.5625 1st Qu.:-0.4820 1st Qu.:0.2839 1st Qu.:-1.049
Median :0.6250 Median :-0.4790 Median :0.2868 Median :-1.041
Mean :0.6250 Mean :-0.4786 Mean :0.2866 Mean :-1.040
3rd Qu.:0.6875 3rd Qu.:-0.4754 3rd Qu.:0.2894 3rd Qu.:-1.032
Max. :0.7500 Max. :-0.4709 Max. :0.2919 Max. :-1.021
ci.ub pvalue sigma2
Min. :0.07924 Min. :0.09342 Min. :0.2542
1st Qu.:0.08111 1st Qu.:0.09407 1st Qu.:0.2571
Median :0.08310 Median :0.09488 Median :0.2594
Mean :0.08321 Mean :0.09498 Mean :0.2591
3rd Qu.:0.08526 3rd Qu.:0.09584 3rd Qu.:0.2612
Max. :0.08756 Max. :0.09692 Max. :0.2629
Level of outcome
Code
##LEVEL OF OUTCOME
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$order_outcome <- as.factor(dataN14_control$order_outcome)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = order_outcome)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.7342949 | 0.4483826 | -0.1445189 | 1.613109 | 0.1014942 | 0.2561155 |
| 0.51 | 0.7331540 | 0.4480922 | -0.1450905 | 1.611398 | 0.1018045 | 0.2563485 |
| 0.52 | 0.7320069 | 0.4477960 | -0.1456571 | 1.609671 | 0.1021144 | 0.2565762 |
| 0.53 | 0.7308532 | 0.4474938 | -0.1462184 | 1.607925 | 0.1024239 | 0.2567983 |
| 0.54 | 0.7296926 | 0.4471852 | -0.1467744 | 1.606159 | 0.1027331 | 0.2570143 |
| 0.55 | 0.7285244 | 0.4468701 | -0.1473249 | 1.604374 | 0.1030419 | 0.2572240 |
| 0.56 | 0.7273481 | 0.4465480 | -0.1478699 | 1.602566 | 0.1033503 | 0.2574270 |
| 0.57 | 0.7261632 | 0.4462186 | -0.1484093 | 1.600736 | 0.1036584 | 0.2576228 |
| 0.58 | 0.7249690 | 0.4458816 | -0.1489429 | 1.598881 | 0.1039661 | 0.2578109 |
| 0.59 | 0.7237647 | 0.4455365 | -0.1494708 | 1.597000 | 0.1042736 | 0.2579909 |
| 0.60 | 0.7225495 | 0.4451829 | -0.1499930 | 1.595092 | 0.1045809 | 0.2581622 |
| 0.61 | 0.7213225 | 0.4448204 | -0.1505094 | 1.593154 | 0.1048881 | 0.2583243 |
| 0.62 | 0.7200826 | 0.4444483 | -0.1510201 | 1.591185 | 0.1051954 | 0.2584765 |
| 0.63 | 0.7188287 | 0.4440663 | -0.1515252 | 1.589183 | 0.1055028 | 0.2586181 |
| 0.64 | 0.7175593 | 0.4436736 | -0.1520250 | 1.587144 | 0.1058106 | 0.2587485 |
| 0.65 | 0.7162730 | 0.4432697 | -0.1525196 | 1.585066 | 0.1061191 | 0.2588667 |
| 0.66 | 0.7149680 | 0.4428538 | -0.1530094 | 1.582945 | 0.1064285 | 0.2589721 |
| 0.67 | 0.7136424 | 0.4424252 | -0.1534949 | 1.580780 | 0.1067393 | 0.2590636 |
| 0.68 | 0.7122940 | 0.4419830 | -0.1539768 | 1.578565 | 0.1070519 | 0.2591402 |
| 0.69 | 0.7109201 | 0.4415264 | -0.1544557 | 1.576296 | 0.1073669 | 0.2592009 |
| 0.70 | 0.7095177 | 0.4410543 | -0.1549329 | 1.573968 | 0.1076851 | 0.2592444 |
| 0.71 | 0.7080834 | 0.4405658 | -0.1554097 | 1.571576 | 0.1080075 | 0.2592694 |
| 0.72 | 0.7066132 | 0.4400596 | -0.1558877 | 1.569114 | 0.1083351 | 0.2592744 |
| 0.73 | 0.7051023 | 0.4395344 | -0.1563693 | 1.566574 | 0.1086694 | 0.2592581 |
| 0.74 | 0.7035452 | 0.4389889 | -0.1568572 | 1.563948 | 0.1090121 | 0.2592185 |
| 0.75 | 0.7019354 | 0.4384216 | -0.1573551 | 1.561226 | 0.1093656 | 0.2591541 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.7019 Min. :0.4384 Min. :-0.1574
1st Qu.:0.5625 1st Qu.:0.7113 1st Qu.:0.4416 1st Qu.:-0.1543
Median :0.6250 Median :0.7195 Median :0.4443 Median :-0.1513
Mean :0.6250 Mean :0.7190 Mean :0.4440 Mean :-0.1511
3rd Qu.:0.6875 3rd Qu.:0.7271 3rd Qu.:0.4465 3rd Qu.:-0.1480
Max. :0.7500 Max. :0.7343 Max. :0.4484 Max. :-0.1445
ci.ub pvalue sigma2
Min. :1.561 Min. :0.1015 Min. :0.2561
1st Qu.:1.577 1st Qu.:0.1034 1st Qu.:0.2575
Median :1.590 Median :0.1053 Median :0.2585
Mean :1.589 Mean :0.1054 Mean :0.2582
3rd Qu.:1.602 3rd Qu.:0.1073 3rd Qu.:0.2592
Max. :1.613 Max. :0.1094 Max. :0.2593
Experiment
Code
## EXPERIMENT #########
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$experiment <- as.factor(dataN14_control$experiment)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = experiment)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.2439203 | 0.4827352 | -1.190064 | 0.7022233 | 0.6133565 | 0.3587093 |
| 0.51 | -0.2433421 | 0.4822426 | -1.188520 | 0.7018360 | 0.6138361 | 0.3586034 |
| 0.52 | -0.2427694 | 0.4817430 | -1.186968 | 0.7014295 | 0.6143038 | 0.3584877 |
| 0.53 | -0.2422023 | 0.4812361 | -1.185408 | 0.7010030 | 0.6147589 | 0.3583619 |
| 0.54 | -0.2416413 | 0.4807214 | -1.183838 | 0.7005554 | 0.6152007 | 0.3582252 |
| 0.55 | -0.2410864 | 0.4801987 | -1.182259 | 0.7000858 | 0.6156285 | 0.3580771 |
| 0.56 | -0.2405381 | 0.4796674 | -1.180669 | 0.6995929 | 0.6160415 | 0.3579167 |
| 0.57 | -0.2399965 | 0.4791272 | -1.179069 | 0.6990755 | 0.6164390 | 0.3577436 |
| 0.58 | -0.2394621 | 0.4785775 | -1.177457 | 0.6985325 | 0.6168200 | 0.3575567 |
| 0.59 | -0.2389352 | 0.4780177 | -1.175833 | 0.6979622 | 0.6171836 | 0.3573553 |
| 0.60 | -0.2384161 | 0.4774472 | -1.174195 | 0.6973633 | 0.6175287 | 0.3571383 |
| 0.61 | -0.2379052 | 0.4768655 | -1.172544 | 0.6967340 | 0.6178543 | 0.3569048 |
| 0.62 | -0.2374028 | 0.4762717 | -1.170878 | 0.6960726 | 0.6181592 | 0.3566535 |
| 0.63 | -0.2369095 | 0.4756651 | -1.169196 | 0.6953770 | 0.6184421 | 0.3563834 |
| 0.64 | -0.2364257 | 0.4750449 | -1.167497 | 0.6946452 | 0.6187016 | 0.3560929 |
| 0.65 | -0.2359519 | 0.4744100 | -1.165778 | 0.6938747 | 0.6189363 | 0.3557805 |
| 0.66 | -0.2354885 | 0.4737595 | -1.164040 | 0.6930631 | 0.6191444 | 0.3554447 |
| 0.67 | -0.2350362 | 0.4730921 | -1.162280 | 0.6922073 | 0.6193241 | 0.3550835 |
| 0.68 | -0.2345955 | 0.4724066 | -1.160495 | 0.6913045 | 0.6194736 | 0.3546950 |
| 0.69 | -0.2341671 | 0.4717016 | -1.158685 | 0.6903510 | 0.6195906 | 0.3542768 |
| 0.70 | -0.2337518 | 0.4709753 | -1.156846 | 0.6893429 | 0.6196727 | 0.3538264 |
| 0.71 | -0.2333502 | 0.4702261 | -1.154977 | 0.6882761 | 0.6197174 | 0.3533409 |
| 0.72 | -0.2329632 | 0.4694519 | -1.153072 | 0.6871456 | 0.6197215 | 0.3528172 |
| 0.73 | -0.2325919 | 0.4686504 | -1.151130 | 0.6859460 | 0.6196817 | 0.3522516 |
| 0.74 | -0.2322372 | 0.4678190 | -1.149146 | 0.6846712 | 0.6195943 | 0.3516401 |
| 0.75 | -0.2319005 | 0.4669547 | -1.147115 | 0.6833140 | 0.6194549 | 0.3509779 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.2439 Min. :0.4670 Min. :-1.190
1st Qu.:0.5625 1st Qu.:-0.2404 1st Qu.:0.4719 1st Qu.:-1.180
Median :0.6250 Median :-0.2372 Median :0.4760 Median :-1.170
Mean :0.6250 Mean :-0.2374 Mean :0.4756 Mean :-1.170
3rd Qu.:0.6875 3rd Qu.:-0.2343 3rd Qu.:0.4795 3rd Qu.:-1.159
Max. :0.7500 Max. :-0.2319 Max. :0.4827 Max. :-1.147
ci.ub pvalue sigma2
Min. :0.6833 Min. :0.6134 Min. :0.3510
1st Qu.:0.6906 1st Qu.:0.6161 1st Qu.:0.3544
Median :0.6957 Median :0.6183 Median :0.3565
Mean :0.6947 Mean :0.6176 Mean :0.3559
3rd Qu.:0.6995 3rd Qu.:0.6195 3rd Qu.:0.3579
Max. :0.7022 Max. :0.6197 Max. :0.3587
System
Code
## SYSTEM ############
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$system <- as.factor(dataN14_control$system_type)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = system)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.2015433 | 0.1904852 | -0.5748875 | 0.1718009 | 0.2900316 | 0.2887986 |
| 0.51 | -0.2011077 | 0.1903924 | -0.5742700 | 0.1720546 | 0.2908403 | 0.2889952 |
| 0.52 | -0.2006746 | 0.1902960 | -0.5736480 | 0.1722987 | 0.2916361 | 0.2891813 |
| 0.53 | -0.2002441 | 0.1901958 | -0.5730211 | 0.1725329 | 0.2924183 | 0.2893561 |
| 0.54 | -0.1998163 | 0.1900916 | -0.5723891 | 0.1727564 | 0.2931861 | 0.2895191 |
| 0.55 | -0.1993914 | 0.1899833 | -0.5717517 | 0.1729689 | 0.2939385 | 0.2896695 |
| 0.56 | -0.1989694 | 0.1898704 | -0.5711087 | 0.1731698 | 0.2946745 | 0.2898067 |
| 0.57 | -0.1985506 | 0.1897530 | -0.5704596 | 0.1733584 | 0.2953932 | 0.2899297 |
| 0.58 | -0.1981352 | 0.1896304 | -0.5698041 | 0.1735336 | 0.2960927 | 0.2900373 |
| 0.59 | -0.1977232 | 0.1895028 | -0.5691419 | 0.1736956 | 0.2967731 | 0.2901296 |
| 0.60 | -0.1973147 | 0.1893697 | -0.5684726 | 0.1738431 | 0.2974325 | 0.2902048 |
| 0.61 | -0.1969102 | 0.1892307 | -0.5677956 | 0.1739752 | 0.2980694 | 0.2902621 |
| 0.62 | -0.1965097 | 0.1890855 | -0.5671105 | 0.1740911 | 0.2986822 | 0.2903001 |
| 0.63 | -0.1961134 | 0.1889337 | -0.5664166 | 0.1741897 | 0.2992693 | 0.2903175 |
| 0.64 | -0.1957217 | 0.1887747 | -0.5657134 | 0.1742699 | 0.2998289 | 0.2903128 |
| 0.65 | -0.1953348 | 0.1886082 | -0.5650000 | 0.1743305 | 0.3003588 | 0.2902844 |
| 0.66 | -0.1949529 | 0.1884336 | -0.5642758 | 0.1743701 | 0.3008570 | 0.2902306 |
| 0.67 | -0.1945763 | 0.1882502 | -0.5635399 | 0.1743873 | 0.3013209 | 0.2901493 |
| 0.68 | -0.1942055 | 0.1880574 | -0.5627912 | 0.1743803 | 0.3017478 | 0.2900384 |
| 0.69 | -0.1938407 | 0.1878545 | -0.5620288 | 0.1743473 | 0.3021348 | 0.2898954 |
| 0.70 | -0.1934824 | 0.1876406 | -0.5612512 | 0.1742864 | 0.3024786 | 0.2897176 |
| 0.71 | -0.1931310 | 0.1874148 | -0.5604572 | 0.1741953 | 0.3027753 | 0.2895019 |
| 0.72 | -0.1927869 | 0.1871760 | -0.5596451 | 0.1740713 | 0.3030209 | 0.2892449 |
| 0.73 | -0.1924508 | 0.1869230 | -0.5588132 | 0.1739115 | 0.3032106 | 0.2889428 |
| 0.74 | -0.1921233 | 0.1866545 | -0.5579594 | 0.1737128 | 0.3033391 | 0.2885912 |
| 0.75 | -0.1918051 | 0.1863689 | -0.5570815 | 0.1734712 | 0.3034002 | 0.2881849 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.2015 Min. :0.1864 Min. :-0.5749
1st Qu.:0.5625 1st Qu.:-0.1989 1st Qu.:0.1879 1st Qu.:-0.5709
Median :0.6250 Median :-0.1963 Median :0.1890 Median :-0.5668
Mean :0.6250 Mean :-0.1964 Mean :0.1888 Mean :-0.5665
3rd Qu.:0.6875 3rd Qu.:-0.1939 3rd Qu.:0.1898 3rd Qu.:-0.5622
Max. :0.7500 Max. :-0.1918 Max. :0.1905 Max. :-0.5571
ci.ub pvalue sigma2
Min. :0.1718 Min. :0.2900 Min. :0.2882
1st Qu.:0.1732 1st Qu.:0.2949 1st Qu.:0.2893
Median :0.1739 Median :0.2990 Median :0.2899
Mean :0.1736 Mean :0.2982 Mean :0.2897
3rd Qu.:0.1743 3rd Qu.:0.3020 3rd Qu.:0.2902
Max. :0.1744 Max. :0.3034 Max. :0.2903
Teacher effects
Code
## TEACHER EFFECTS ########
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$teacher <- as.factor(dataN14_control$teacher_effects)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = teacher)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.3780995 | 0.2389033 | -0.8463414 | 0.0901423 | 0.1135020 | 0.2239991 |
| 0.51 | -0.3773235 | 0.2387537 | -0.8452721 | 0.0906251 | 0.1140180 | 0.2242824 |
| 0.52 | -0.3765474 | 0.2385991 | -0.8441931 | 0.0910983 | 0.1145291 | 0.2245574 |
| 0.53 | -0.3757712 | 0.2384395 | -0.8431040 | 0.0915616 | 0.1150349 | 0.2248239 |
| 0.54 | -0.3749949 | 0.2382745 | -0.8420042 | 0.0920145 | 0.1155351 | 0.2250812 |
| 0.55 | -0.3742184 | 0.2381038 | -0.8408933 | 0.0924565 | 0.1160293 | 0.2253288 |
| 0.56 | -0.3734417 | 0.2379273 | -0.8397706 | 0.0928871 | 0.1165169 | 0.2255661 |
| 0.57 | -0.3726649 | 0.2377445 | -0.8386355 | 0.0933058 | 0.1169976 | 0.2257925 |
| 0.58 | -0.3718877 | 0.2375552 | -0.8374874 | 0.0937120 | 0.1174707 | 0.2260073 |
| 0.59 | -0.3711102 | 0.2373590 | -0.8363253 | 0.0941049 | 0.1179357 | 0.2262097 |
| 0.60 | -0.3703323 | 0.2371555 | -0.8351486 | 0.0944839 | 0.1183919 | 0.2263989 |
| 0.61 | -0.3695540 | 0.2369443 | -0.8339562 | 0.0948482 | 0.1188387 | 0.2265740 |
| 0.62 | -0.3687752 | 0.2367248 | -0.8327473 | 0.0951968 | 0.1192753 | 0.2267340 |
| 0.63 | -0.3679958 | 0.2364966 | -0.8315206 | 0.0955289 | 0.1197009 | 0.2268777 |
| 0.64 | -0.3672158 | 0.2362590 | -0.8302751 | 0.0958434 | 0.1201145 | 0.2270041 |
| 0.65 | -0.3664351 | 0.2360116 | -0.8290093 | 0.0961390 | 0.1205151 | 0.2271117 |
| 0.66 | -0.3656536 | 0.2357534 | -0.8277217 | 0.0964145 | 0.1209016 | 0.2271990 |
| 0.67 | -0.3648712 | 0.2354838 | -0.8264109 | 0.0966685 | 0.1212727 | 0.2272645 |
| 0.68 | -0.3640876 | 0.2352020 | -0.8250750 | 0.0968998 | 0.1216274 | 0.2273067 |
| 0.69 | -0.3633032 | 0.2349067 | -0.8237117 | 0.0971054 | 0.1219633 | 0.2273227 |
| 0.70 | -0.3625175 | 0.2345969 | -0.8223189 | 0.0972839 | 0.1222789 | 0.2273106 |
| 0.71 | -0.3617305 | 0.2342714 | -0.8208940 | 0.0974329 | 0.1225723 | 0.2272678 |
| 0.72 | -0.3609422 | 0.2339288 | -0.8194342 | 0.0975497 | 0.1228410 | 0.2271914 |
| 0.73 | -0.3601525 | 0.2335674 | -0.8179361 | 0.0976312 | 0.1230824 | 0.2270780 |
| 0.74 | -0.3593612 | 0.2331853 | -0.8163960 | 0.0976737 | 0.1232932 | 0.2269238 |
| 0.75 | -0.3585684 | 0.2327805 | -0.8148098 | 0.0976730 | 0.1234699 | 0.2267244 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.3781 Min. :0.2328 Min. :-0.8463
1st Qu.:0.5625 1st Qu.:-0.3732 1st Qu.:0.2350 1st Qu.:-0.8395
Median :0.6250 Median :-0.3684 Median :0.2366 Median :-0.8321
Mean :0.6250 Mean :-0.3684 Mean :0.2363 Mean :-0.8316
3rd Qu.:0.6875 3rd Qu.:-0.3635 3rd Qu.:0.2379 3rd Qu.:-0.8241
Max. :0.7500 Max. :-0.3586 Max. :0.2389 Max. :-0.8148
ci.ub pvalue sigma2
Min. :0.09014 Min. :0.1135 Min. :0.2240
1st Qu.:0.09299 1st Qu.:0.1166 1st Qu.:0.2256
Median :0.09536 Median :0.1195 Median :0.2267
Mean :0.09486 Mean :0.1191 Mean :0.2263
3rd Qu.:0.09705 3rd Qu.:0.1219 3rd Qu.:0.2272
Max. :0.09767 Max. :0.1235 Max. :0.2273
Validated tool
Code
## VALIDATED TOOL ###############
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$validated_tool <- as.factor(dataN14_control$validated_tool)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = validated_tool)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 1.114876 | 0.4352933 | 0.2617172 | 1.968036 | 0.0104309 | 0.1519381 |
| 0.51 | 1.114375 | 0.4349179 | 0.2619516 | 1.966798 | 0.0103992 | 0.1518810 |
| 0.52 | 1.113866 | 0.4345358 | 0.2621918 | 1.965541 | 0.0103668 | 0.1518193 |
| 0.53 | 1.113350 | 0.4341465 | 0.2624383 | 1.964261 | 0.0103338 | 0.1517529 |
| 0.54 | 1.112825 | 0.4337496 | 0.2626917 | 1.962959 | 0.0102999 | 0.1516812 |
| 0.55 | 1.112292 | 0.4333444 | 0.2629527 | 1.961632 | 0.0102653 | 0.1516040 |
| 0.56 | 1.111750 | 0.4329305 | 0.2632221 | 1.960278 | 0.0102298 | 0.1515208 |
| 0.57 | 1.111199 | 0.4325070 | 0.2635006 | 1.958897 | 0.0101932 | 0.1514310 |
| 0.58 | 1.110638 | 0.4320738 | 0.2637886 | 1.957486 | 0.0101557 | 0.1513346 |
| 0.59 | 1.110066 | 0.4316291 | 0.2640884 | 1.956043 | 0.0101170 | 0.1512303 |
| 0.60 | 1.109483 | 0.4311725 | 0.2644007 | 1.954566 | 0.0100770 | 0.1511177 |
| 0.61 | 1.108889 | 0.4307031 | 0.2647266 | 1.953052 | 0.0100355 | 0.1509963 |
| 0.62 | 1.108283 | 0.4302197 | 0.2650677 | 1.951498 | 0.0099926 | 0.1508652 |
| 0.63 | 1.107663 | 0.4297211 | 0.2654257 | 1.949901 | 0.0099479 | 0.1507235 |
| 0.64 | 1.107031 | 0.4292059 | 0.2658023 | 1.948259 | 0.0099014 | 0.1505703 |
| 0.65 | 1.106383 | 0.4286727 | 0.2661999 | 1.946566 | 0.0098529 | 0.1504045 |
| 0.66 | 1.105720 | 0.4281197 | 0.2666208 | 1.944819 | 0.0098020 | 0.1502249 |
| 0.67 | 1.105041 | 0.4275451 | 0.2670678 | 1.943014 | 0.0097487 | 0.1500301 |
| 0.68 | 1.104344 | 0.4269463 | 0.2675449 | 1.941143 | 0.0096925 | 0.1498184 |
| 0.69 | 1.103629 | 0.4263216 | 0.2680540 | 1.939204 | 0.0096332 | 0.1495886 |
| 0.70 | 1.102894 | 0.4256679 | 0.2686002 | 1.937188 | 0.0095705 | 0.1493385 |
| 0.71 | 1.102138 | 0.4249821 | 0.2691883 | 1.935088 | 0.0095039 | 0.1490659 |
| 0.72 | 1.101359 | 0.4242607 | 0.2698239 | 1.932895 | 0.0094330 | 0.1487682 |
| 0.73 | 1.100557 | 0.4234994 | 0.2705133 | 1.930600 | 0.0093572 | 0.1484427 |
| 0.74 | 1.099729 | 0.4226936 | 0.2712643 | 1.928193 | 0.0092759 | 0.1480860 |
| 0.75 | 1.098872 | 0.4218378 | 0.2720857 | 1.925659 | 0.0091884 | 0.1476943 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :1.099 Min. :0.4218 Min. :0.2617
1st Qu.:0.5625 1st Qu.:1.104 1st Qu.:0.4265 1st Qu.:0.2633
Median :0.6250 Median :1.108 Median :0.4300 Median :0.2652
Mean :0.6250 Mean :1.108 Mean :0.4295 Mean :0.2658
3rd Qu.:0.6875 3rd Qu.:1.112 3rd Qu.:0.4328 3rd Qu.:0.2679
Max. :0.7500 Max. :1.115 Max. :0.4353 Max. :0.2721
ci.ub pvalue sigma2
Min. :1.926 Min. :0.009188 Min. :0.1477
1st Qu.:1.940 1st Qu.:0.009648 1st Qu.:0.1496
Median :1.951 Median :0.009970 Median :0.1508
Mean :1.949 Mean :0.009916 Mean :0.1505
3rd Qu.:1.960 3rd Qu.:0.010221 3rd Qu.:0.1515
Max. :1.968 Max. :0.010431 Max. :0.1519
Reliability of measure
Code
## RELIABILITY MEASURE ###########
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$reliability <- as.factor(dataN14_control$reliability_measurement)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = reliability)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.6580934 | 0.2236447 | -1.096429 | -0.2197579 | 0.0032548 | 0.1374712 |
| 0.51 | -0.6568891 | 0.2241825 | -1.096279 | -0.2174996 | 0.0033879 | 0.1385670 |
| 0.52 | -0.6556745 | 0.2247178 | -1.096113 | -0.2152356 | 0.0035255 | 0.1396651 |
| 0.53 | -0.6544488 | 0.2252503 | -1.095931 | -0.2129664 | 0.0036675 | 0.1407648 |
| 0.54 | -0.6532114 | 0.2257803 | -1.095733 | -0.2106901 | 0.0038143 | 0.1418669 |
| 0.55 | -0.6519615 | 0.2263078 | -1.095517 | -0.2084064 | 0.0039659 | 0.1429710 |
| 0.56 | -0.6506982 | 0.2268327 | -1.095282 | -0.2061142 | 0.0041226 | 0.1440771 |
| 0.57 | -0.6494207 | 0.2273552 | -1.095029 | -0.2038127 | 0.0042845 | 0.1451853 |
| 0.58 | -0.6481277 | 0.2278751 | -1.094755 | -0.2015008 | 0.0044520 | 0.1462955 |
| 0.59 | -0.6468184 | 0.2283925 | -1.094459 | -0.1991773 | 0.0046251 | 0.1474079 |
| 0.60 | -0.6454914 | 0.2289075 | -1.094142 | -0.1968410 | 0.0048042 | 0.1485223 |
| 0.61 | -0.6441453 | 0.2294200 | -1.093800 | -0.1944903 | 0.0049895 | 0.1496388 |
| 0.62 | -0.6427787 | 0.2299302 | -1.093434 | -0.1921237 | 0.0051813 | 0.1507574 |
| 0.63 | -0.6413899 | 0.2304381 | -1.093040 | -0.1897395 | 0.0053801 | 0.1518782 |
| 0.64 | -0.6399771 | 0.2309437 | -1.092618 | -0.1873357 | 0.0055861 | 0.1530013 |
| 0.65 | -0.6385381 | 0.2314471 | -1.092166 | -0.1849102 | 0.0057997 | 0.1541267 |
| 0.66 | -0.6370708 | 0.2319484 | -1.091681 | -0.1824603 | 0.0060215 | 0.1552546 |
| 0.67 | -0.6355726 | 0.2324477 | -1.091162 | -0.1799835 | 0.0062520 | 0.1563850 |
| 0.68 | -0.6340405 | 0.2329450 | -1.090604 | -0.1774766 | 0.0064918 | 0.1575181 |
| 0.69 | -0.6324713 | 0.2334408 | -1.090007 | -0.1749357 | 0.0067417 | 0.1586544 |
| 0.70 | -0.6308613 | 0.2339349 | -1.089365 | -0.1723573 | 0.0070022 | 0.1597937 |
| 0.71 | -0.6292064 | 0.2344277 | -1.088676 | -0.1697366 | 0.0072745 | 0.1609365 |
| 0.72 | -0.6275016 | 0.2349193 | -1.087935 | -0.1670682 | 0.0075595 | 0.1620833 |
| 0.73 | -0.6257416 | 0.2354103 | -1.087137 | -0.1643458 | 0.0078585 | 0.1632346 |
| 0.74 | -0.6239199 | 0.2359011 | -1.086278 | -0.1615622 | 0.0081730 | 0.1643911 |
| 0.75 | -0.6220291 | 0.2363922 | -1.085349 | -0.1587089 | 0.0085048 | 0.1655538 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.6581 Min. :0.2236 Min. :-1.096
1st Qu.:0.5625 1st Qu.:-0.6504 1st Qu.:0.2270 1st Qu.:-1.095
Median :0.6250 Median :-0.6421 Median :0.2302 Median :-1.093
Mean :0.6250 Mean :-0.6414 Mean :0.2301 Mean :-1.092
3rd Qu.:0.6875 3rd Qu.:-0.6329 3rd Qu.:0.2333 3rd Qu.:-1.090
Max. :0.7500 Max. :-0.6220 Max. :0.2364 Max. :-1.085
ci.ub pvalue sigma2
Min. :-0.2198 Min. :0.003255 Min. :0.1375
1st Qu.:-0.2055 1st Qu.:0.004163 1st Qu.:0.1444
Median :-0.1909 Median :0.005281 Median :0.1513
Mean :-0.1904 Mean :0.005489 Mean :0.1514
3rd Qu.:-0.1756 3rd Qu.:0.006679 3rd Qu.:0.1584
Max. :-0.1587 Max. :0.008505 Max. :0.1656
Code
#post-hoc:
#low
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$reliability_low <- ifelse(dataN14_control$reliability_measurement == 0, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = reliability_low)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.8935590 | 0.5604653 | -0.2049329 | 1.992051 | 0.1108652 | 0.2626922 |
| 0.51 | 0.8926921 | 0.5604341 | -0.2057386 | 1.991123 | 0.1111920 | 0.2628427 |
| 0.52 | 0.8918096 | 0.5603991 | -0.2065525 | 1.990172 | 0.1115234 | 0.2629897 |
| 0.53 | 0.8909106 | 0.5603602 | -0.2073752 | 1.989196 | 0.1118598 | 0.2631331 |
| 0.54 | 0.8899938 | 0.5603170 | -0.2082074 | 1.988195 | 0.1122015 | 0.2632726 |
| 0.55 | 0.8890581 | 0.5602695 | -0.2090500 | 1.987166 | 0.1125489 | 0.2634081 |
| 0.56 | 0.8881021 | 0.5602174 | -0.2099039 | 1.986108 | 0.1129025 | 0.2635394 |
| 0.57 | 0.8871243 | 0.5601604 | -0.2107699 | 1.985019 | 0.1132627 | 0.2636661 |
| 0.58 | 0.8861232 | 0.5600983 | -0.2116493 | 1.983896 | 0.1136301 | 0.2637881 |
| 0.59 | 0.8850969 | 0.5600307 | -0.2125431 | 1.982737 | 0.1140053 | 0.2639051 |
| 0.60 | 0.8840434 | 0.5599574 | -0.2134530 | 1.981540 | 0.1143891 | 0.2640168 |
| 0.61 | 0.8829607 | 0.5598781 | -0.2143803 | 1.980302 | 0.1147820 | 0.2641228 |
| 0.62 | 0.8818461 | 0.5597924 | -0.2153269 | 1.979019 | 0.1151851 | 0.2642230 |
| 0.63 | 0.8806971 | 0.5597000 | -0.2162947 | 1.977689 | 0.1155993 | 0.2643168 |
| 0.64 | 0.8795105 | 0.5596004 | -0.2172861 | 1.976307 | 0.1160257 | 0.2644040 |
| 0.65 | 0.8782831 | 0.5594932 | -0.2183035 | 1.974870 | 0.1164656 | 0.2644841 |
| 0.66 | 0.8770108 | 0.5593780 | -0.2193500 | 1.973372 | 0.1169203 | 0.2645568 |
| 0.67 | 0.8756895 | 0.5592543 | -0.2204288 | 1.971808 | 0.1173916 | 0.2646215 |
| 0.68 | 0.8743143 | 0.5591217 | -0.2215440 | 1.970173 | 0.1178814 | 0.2646779 |
| 0.69 | 0.8728796 | 0.5589794 | -0.2227000 | 1.968459 | 0.1183918 | 0.2647253 |
| 0.70 | 0.8713791 | 0.5588271 | -0.2239019 | 1.966660 | 0.1189254 | 0.2647633 |
| 0.71 | 0.8698055 | 0.5586640 | -0.2251559 | 1.964767 | 0.1194851 | 0.2647912 |
| 0.72 | 0.8681505 | 0.5584896 | -0.2264691 | 1.962770 | 0.1200744 | 0.2648086 |
| 0.73 | 0.8664044 | 0.5583027 | -0.2278488 | 1.960658 | 0.1206971 | 0.2648143 |
| 0.74 | 0.8645561 | 0.5581036 | -0.2293069 | 1.958419 | 0.1213586 | 0.2648087 |
| 0.75 | 0.8625924 | 0.5578911 | -0.2308541 | 1.956039 | 0.1220644 | 0.2647906 |
Code
summary(sensitivity_moderator) #mean = .88, p = .116 ri_t beta se ci.lb
Min. :0.5000 Min. :0.8626 Min. :0.5579 Min. :-0.2309
1st Qu.:0.5625 1st Qu.:0.8732 1st Qu.:0.5590 1st Qu.:-0.2224
Median :0.6250 Median :0.8813 Median :0.5597 Median :-0.2158
Mean :0.6250 Mean :0.8802 Mean :0.5595 Mean :-0.2165
3rd Qu.:0.6875 3rd Qu.:0.8879 3rd Qu.:0.5602 3rd Qu.:-0.2101
Max. :0.7500 Max. :0.8936 Max. :0.5605 Max. :-0.2049
ci.ub pvalue sigma2
Min. :1.956 Min. :0.1109 Min. :0.2627
1st Qu.:1.969 1st Qu.:0.1130 1st Qu.:0.2636
Median :1.978 Median :0.1154 Median :0.2643
Mean :1.977 Mean :0.1158 Mean :0.2641
3rd Qu.:1.986 3rd Qu.:0.1183 3rd Qu.:0.2647
Max. :1.992 Max. :0.1221 Max. :0.2648
Code
#high
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$reliability_high <- ifelse(dataN14_control$reliability_measurement == 1, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = reliability_high)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.3136996 | 0.4742084 | -0.6157317 | 1.243131 | 0.5082771 | 0.3845022 |
| 0.51 | 0.3133977 | 0.4736375 | -0.6149147 | 1.241710 | 0.5081747 | 0.3842278 |
| 0.52 | 0.3130869 | 0.4730617 | -0.6140969 | 1.240271 | 0.5080795 | 0.3839466 |
| 0.53 | 0.3127670 | 0.4724807 | -0.6132781 | 1.238812 | 0.5079918 | 0.3836582 |
| 0.54 | 0.3124376 | 0.4718942 | -0.6124581 | 1.237333 | 0.5079120 | 0.3833621 |
| 0.55 | 0.3120982 | 0.4713020 | -0.6116366 | 1.235833 | 0.5078402 | 0.3830579 |
| 0.56 | 0.3117485 | 0.4707036 | -0.6108136 | 1.234311 | 0.5077769 | 0.3827449 |
| 0.57 | 0.3113880 | 0.4700988 | -0.6099887 | 1.232765 | 0.5077223 | 0.3824226 |
| 0.58 | 0.3110163 | 0.4694871 | -0.6091616 | 1.231194 | 0.5076767 | 0.3820903 |
| 0.59 | 0.3106327 | 0.4688681 | -0.6083319 | 1.229597 | 0.5076405 | 0.3817472 |
| 0.60 | 0.3102368 | 0.4682414 | -0.6074994 | 1.227973 | 0.5076141 | 0.3813927 |
| 0.61 | 0.3098279 | 0.4676063 | -0.6066636 | 1.226319 | 0.5075978 | 0.3810258 |
| 0.62 | 0.3094055 | 0.4669624 | -0.6058239 | 1.224635 | 0.5075919 | 0.3806456 |
| 0.63 | 0.3089689 | 0.4663090 | -0.6049799 | 1.222918 | 0.5075970 | 0.3802510 |
| 0.64 | 0.3085174 | 0.4656454 | -0.6041308 | 1.221166 | 0.5076133 | 0.3798408 |
| 0.65 | 0.3080501 | 0.4649709 | -0.6032760 | 1.219376 | 0.5076413 | 0.3794139 |
| 0.66 | 0.3075664 | 0.4642845 | -0.6024145 | 1.217547 | 0.5076813 | 0.3789687 |
| 0.67 | 0.3070652 | 0.4635854 | -0.6015455 | 1.215676 | 0.5077339 | 0.3785037 |
| 0.68 | 0.3065457 | 0.4628725 | -0.6006678 | 1.213759 | 0.5077994 | 0.3780171 |
| 0.69 | 0.3060067 | 0.4621446 | -0.5997801 | 1.211794 | 0.5078783 | 0.3775070 |
| 0.70 | 0.3054473 | 0.4614004 | -0.5988810 | 1.209775 | 0.5079710 | 0.3769711 |
| 0.71 | 0.3048660 | 0.4606385 | -0.5979688 | 1.207701 | 0.5080779 | 0.3764070 |
| 0.72 | 0.3042617 | 0.4598573 | -0.5970420 | 1.205566 | 0.5081995 | 0.3758125 |
| 0.73 | 0.3036328 | 0.4590545 | -0.5960974 | 1.203363 | 0.5083361 | 0.3751831 |
| 0.74 | 0.3029776 | 0.4582281 | -0.5951330 | 1.201088 | 0.5084880 | 0.3745160 |
| 0.75 | 0.3022943 | 0.4573757 | -0.5941456 | 1.198734 | 0.5086559 | 0.3738070 |
Code
summary(sensitivity_moderator) #mean = .31, p = .508 ri_t beta se ci.lb
Min. :0.5000 Min. :0.3023 Min. :0.4574 Min. :-0.6157
1st Qu.:0.5625 1st Qu.:0.3061 1st Qu.:0.4623 1st Qu.:-0.6106
Median :0.6250 Median :0.3092 Median :0.4666 Median :-0.6054
Mean :0.6250 Mean :0.3088 Mean :0.4663 Mean :-0.6052
3rd Qu.:0.6875 3rd Qu.:0.3117 3rd Qu.:0.4706 3rd Qu.:-0.6000
Max. :0.7500 Max. :0.3137 Max. :0.4742 Max. :-0.5941
ci.ub pvalue sigma2
Min. :1.199 Min. :0.5076 Min. :0.3738
1st Qu.:1.212 1st Qu.:0.5077 1st Qu.:0.3776
Median :1.224 Median :0.5078 Median :0.3804
Mean :1.223 Mean :0.5079 Mean :0.3800
3rd Qu.:1.234 3rd Qu.:0.5081 3rd Qu.:0.3827
Max. :1.243 Max. :0.5087 Max. :0.3845
Code
#not specified
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$reliability_not <- ifelse(dataN14_control$reliability_measurement == 2, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = reliability_not)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.8332949 | 0.3858626 | -1.589572 | -0.0770181 | 0.0308064 | 0.2086035 |
| 0.51 | -0.8313473 | 0.3858681 | -1.587635 | -0.0750597 | 0.0312021 | 0.2093181 |
| 0.52 | -0.8293853 | 0.3858724 | -1.585681 | -0.0730892 | 0.0316045 | 0.2100339 |
| 0.53 | -0.8274080 | 0.3858752 | -1.583710 | -0.0711065 | 0.0320139 | 0.2107506 |
| 0.54 | -0.8254147 | 0.3858763 | -1.581718 | -0.0691111 | 0.0324304 | 0.2114680 |
| 0.55 | -0.8234043 | 0.3858755 | -1.579706 | -0.0671022 | 0.0328542 | 0.2121859 |
| 0.56 | -0.8213760 | 0.3858729 | -1.577673 | -0.0650791 | 0.0332859 | 0.2129042 |
| 0.57 | -0.8193285 | 0.3858681 | -1.575616 | -0.0630409 | 0.0337255 | 0.2136226 |
| 0.58 | -0.8172607 | 0.3858611 | -1.573535 | -0.0609868 | 0.0341736 | 0.2143409 |
| 0.59 | -0.8151712 | 0.3858517 | -1.571427 | -0.0589158 | 0.0346304 | 0.2150590 |
| 0.60 | -0.8130586 | 0.3858396 | -1.569290 | -0.0568268 | 0.0350964 | 0.2157766 |
| 0.61 | -0.8109214 | 0.3858248 | -1.567124 | -0.0547187 | 0.0355720 | 0.2164933 |
| 0.62 | -0.8087577 | 0.3858069 | -1.564925 | -0.0525901 | 0.0360577 | 0.2172090 |
| 0.63 | -0.8065657 | 0.3857857 | -1.562692 | -0.0504396 | 0.0365541 | 0.2179233 |
| 0.64 | -0.8043432 | 0.3857609 | -1.560421 | -0.0482656 | 0.0370619 | 0.2186357 |
| 0.65 | -0.8020879 | 0.3857323 | -1.558109 | -0.0460664 | 0.0375815 | 0.2193461 |
| 0.66 | -0.7997972 | 0.3856995 | -1.555754 | -0.0438400 | 0.0381139 | 0.2200538 |
| 0.67 | -0.7974681 | 0.3856622 | -1.553352 | -0.0415842 | 0.0386598 | 0.2207585 |
| 0.68 | -0.7950975 | 0.3856199 | -1.550898 | -0.0392964 | 0.0392203 | 0.2214596 |
| 0.69 | -0.7926815 | 0.3855722 | -1.548389 | -0.0369739 | 0.0397963 | 0.2221565 |
| 0.70 | -0.7902162 | 0.3855187 | -1.545819 | -0.0346135 | 0.0403890 | 0.2228488 |
| 0.71 | -0.7876969 | 0.3854588 | -1.543182 | -0.0322114 | 0.0410000 | 0.2235355 |
| 0.72 | -0.7851181 | 0.3853920 | -1.540473 | -0.0297636 | 0.0416307 | 0.2242162 |
| 0.73 | -0.7824739 | 0.3853177 | -1.537683 | -0.0272651 | 0.0422830 | 0.2248898 |
| 0.74 | -0.7797571 | 0.3852351 | -1.534804 | -0.0247102 | 0.0429591 | 0.2255556 |
| 0.75 | -0.7769598 | 0.3851436 | -1.531827 | -0.0220921 | 0.0436616 | 0.2262127 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.8333 Min. :0.3851 Min. :-1.590
1st Qu.:0.5625 1st Qu.:-0.8209 1st Qu.:0.3856 1st Qu.:-1.577
Median :0.6250 Median :-0.8077 Median :0.3858 Median :-1.564
Mean :0.6250 Mean :-0.8068 Mean :0.3857 Mean :-1.563
3rd Qu.:0.6875 3rd Qu.:-0.7933 3rd Qu.:0.3859 3rd Qu.:-1.549
Max. :0.7500 Max. :-0.7770 Max. :0.3859 Max. :-1.532
ci.ub pvalue sigma2
Min. :-0.07702 Min. :0.03081 Min. :0.2086
1st Qu.:-0.06457 1st Qu.:0.03340 1st Qu.:0.2131
Median :-0.05151 Median :0.03631 Median :0.2176
Mean :-0.05084 Mean :0.03663 Mean :0.2175
3rd Qu.:-0.03755 3rd Qu.:0.03965 3rd Qu.:0.2220
Max. :-0.02209 Max. :0.04366 Max. :0.2262
Treatment fidelity
Code
## TREATMENT FIDELITY ###########
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$fidelity <- as.factor(dataN14_control$treatment_fidelity)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = fidelity)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.4845101 | 0.1776556 | 0.1363115 | 0.8327087 | 0.0063866 | 0.1589562 |
| 0.51 | 0.4839456 | 0.1775103 | 0.1360319 | 0.8318593 | 0.0064049 | 0.1591700 |
| 0.52 | 0.4833787 | 0.1773619 | 0.1357558 | 0.8310016 | 0.0064227 | 0.1593789 |
| 0.53 | 0.4828092 | 0.1772102 | 0.1354835 | 0.8301349 | 0.0064399 | 0.1595825 |
| 0.54 | 0.4822370 | 0.1770551 | 0.1352154 | 0.8292586 | 0.0064564 | 0.1597804 |
| 0.55 | 0.4816618 | 0.1768965 | 0.1349510 | 0.8283726 | 0.0064722 | 0.1599726 |
| 0.56 | 0.4810834 | 0.1767341 | 0.1346909 | 0.8274760 | 0.0064874 | 0.1601585 |
| 0.57 | 0.4805016 | 0.1765678 | 0.1344350 | 0.8265682 | 0.0065017 | 0.1603378 |
| 0.58 | 0.4799161 | 0.1763974 | 0.1341836 | 0.8256486 | 0.0065153 | 0.1605099 |
| 0.59 | 0.4793266 | 0.1762225 | 0.1339368 | 0.8247164 | 0.0065280 | 0.1606744 |
| 0.60 | 0.4787327 | 0.1760430 | 0.1336947 | 0.8237707 | 0.0065399 | 0.1608308 |
| 0.61 | 0.4781341 | 0.1758586 | 0.1334575 | 0.8228106 | 0.0065508 | 0.1609785 |
| 0.62 | 0.4775302 | 0.1756690 | 0.1332254 | 0.8218351 | 0.0065608 | 0.1611169 |
| 0.63 | 0.4769207 | 0.1754738 | 0.1329984 | 0.8208431 | 0.0065697 | 0.1612455 |
| 0.64 | 0.4763050 | 0.1752728 | 0.1327767 | 0.8198334 | 0.0065776 | 0.1613634 |
| 0.65 | 0.4756825 | 0.1750655 | 0.1325605 | 0.8188045 | 0.0065843 | 0.1614699 |
| 0.66 | 0.4750525 | 0.1748515 | 0.1323499 | 0.8177551 | 0.0065898 | 0.1615642 |
| 0.67 | 0.4744142 | 0.1746304 | 0.1321449 | 0.8166835 | 0.0065941 | 0.1616453 |
| 0.68 | 0.4737667 | 0.1744017 | 0.1319457 | 0.8155877 | 0.0065971 | 0.1617122 |
| 0.69 | 0.4731089 | 0.1741648 | 0.1317523 | 0.8144656 | 0.0065987 | 0.1617640 |
| 0.70 | 0.4724399 | 0.1739191 | 0.1315646 | 0.8133151 | 0.0065989 | 0.1617992 |
| 0.71 | 0.4717580 | 0.1736641 | 0.1313827 | 0.8121334 | 0.0065977 | 0.1618167 |
| 0.72 | 0.4710620 | 0.1733989 | 0.1312064 | 0.8109175 | 0.0065949 | 0.1618150 |
| 0.73 | 0.4703499 | 0.1731227 | 0.1310356 | 0.8096642 | 0.0065905 | 0.1617924 |
| 0.74 | 0.4696196 | 0.1728347 | 0.1308698 | 0.8083695 | 0.0065845 | 0.1617473 |
| 0.75 | 0.4688689 | 0.1725340 | 0.1307085 | 0.8070292 | 0.0065768 | 0.1616776 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.4689 Min. :0.1725 Min. :0.1307
1st Qu.:0.5625 1st Qu.:0.4733 1st Qu.:0.1742 1st Qu.:0.1318
Median :0.6250 Median :0.4772 Median :0.1756 Median :0.1331
Mean :0.6250 Mean :0.4770 Mean :0.1754 Mean :0.1333
3rd Qu.:0.6875 3rd Qu.:0.4809 3rd Qu.:0.1767 3rd Qu.:0.1346
Max. :0.7500 Max. :0.4845 Max. :0.1777 Max. :0.1363
ci.ub pvalue sigma2
Min. :0.8070 Min. :0.006387 Min. :0.1590
1st Qu.:0.8147 1st Qu.:0.006491 1st Qu.:0.1602
Median :0.8213 Median :0.006565 Median :0.1612
Mean :0.8208 Mean :0.006535 Mean :0.1609
3rd Qu.:0.8272 3rd Qu.:0.006590 3rd Qu.:0.1617
Max. :0.8327 Max. :0.006599 Max. :0.1618
Code
#post-hoc:
#not specified
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$fidelity_not <- ifelse(dataN14_control$treatment_fidelity == 0, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = fidelity_not)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | -0.9490237 | 0.3387597 | -1.612980 | -0.2850669 | 0.0050871 | 0.1550865 |
| 0.51 | -0.9481592 | 0.3383239 | -1.611262 | -0.2850564 | 0.0050705 | 0.1551256 |
| 0.52 | -0.9472939 | 0.3378794 | -1.609525 | -0.2850626 | 0.0050528 | 0.1551565 |
| 0.53 | -0.9464279 | 0.3374255 | -1.607770 | -0.2850861 | 0.0050340 | 0.1551787 |
| 0.54 | -0.9455609 | 0.3369619 | -1.605994 | -0.2851278 | 0.0050140 | 0.1551916 |
| 0.55 | -0.9446929 | 0.3364881 | -1.604197 | -0.2851884 | 0.0049926 | 0.1551949 |
| 0.56 | -0.9438236 | 0.3360034 | -1.602378 | -0.2852690 | 0.0049700 | 0.1551878 |
| 0.57 | -0.9429529 | 0.3355074 | -1.600535 | -0.2853704 | 0.0049460 | 0.1551699 |
| 0.58 | -0.9420805 | 0.3349994 | -1.598667 | -0.2854937 | 0.0049206 | 0.1551404 |
| 0.59 | -0.9412062 | 0.3344785 | -1.596772 | -0.2856404 | 0.0048937 | 0.1550985 |
| 0.60 | -0.9403299 | 0.3339444 | -1.594849 | -0.2858108 | 0.0048652 | 0.1550437 |
| 0.61 | -0.9394510 | 0.3333960 | -1.592895 | -0.2860068 | 0.0048350 | 0.1549751 |
| 0.62 | -0.9385694 | 0.3328325 | -1.590909 | -0.2862296 | 0.0048032 | 0.1548917 |
| 0.63 | -0.9376845 | 0.3322529 | -1.588888 | -0.2864808 | 0.0047695 | 0.1547925 |
| 0.64 | -0.9367961 | 0.3316561 | -1.586830 | -0.2867620 | 0.0047340 | 0.1546765 |
| 0.65 | -0.9359034 | 0.3310410 | -1.584732 | -0.2870749 | 0.0046964 | 0.1545425 |
| 0.66 | -0.9350061 | 0.3304063 | -1.582591 | -0.2874216 | 0.0046567 | 0.1543890 |
| 0.67 | -0.9341033 | 0.3297506 | -1.580403 | -0.2878040 | 0.0046148 | 0.1542148 |
| 0.68 | -0.9331944 | 0.3290723 | -1.578164 | -0.2882246 | 0.0045706 | 0.1540182 |
| 0.69 | -0.9322785 | 0.3283696 | -1.575871 | -0.2886858 | 0.0045239 | 0.1537974 |
| 0.70 | -0.9313545 | 0.3276407 | -1.573519 | -0.2891905 | 0.0044746 | 0.1535505 |
| 0.71 | -0.9304214 | 0.3268833 | -1.571101 | -0.2897418 | 0.0044225 | 0.1532753 |
| 0.72 | -0.9294777 | 0.3260950 | -1.568612 | -0.2903432 | 0.0043674 | 0.1529693 |
| 0.73 | -0.9285221 | 0.3252731 | -1.566046 | -0.2909986 | 0.0043092 | 0.1526298 |
| 0.74 | -0.9275528 | 0.3244144 | -1.563393 | -0.2917123 | 0.0042476 | 0.1522536 |
| 0.75 | -0.9265677 | 0.3235154 | -1.560646 | -0.2924893 | 0.0041825 | 0.1518372 |
Code
summary(sensitivity_moderator) #mean = -.94, p = .005 ri_t beta se ci.lb
Min. :0.5000 Min. :-0.9490 Min. :0.3235 Min. :-1.613
1st Qu.:0.5625 1st Qu.:-0.9436 1st Qu.:0.3285 1st Qu.:-1.602
Median :0.6250 Median :-0.9381 Median :0.3325 Median :-1.590
Mean :0.6250 Mean :-0.9380 Mean :0.3321 Mean :-1.589
3rd Qu.:0.6875 3rd Qu.:-0.9325 3rd Qu.:0.3359 3rd Qu.:-1.576
Max. :0.7500 Max. :-0.9266 Max. :0.3388 Max. :-1.561
ci.ub pvalue sigma2
Min. :-0.2925 Min. :0.004182 Min. :0.1518
1st Qu.:-0.2886 1st Qu.:0.004536 1st Qu.:0.1539
Median :-0.2864 Median :0.004786 Median :0.1548
Mean :-0.2872 Mean :0.004733 Mean :0.1544
3rd Qu.:-0.2853 3rd Qu.:0.004964 3rd Qu.:0.1551
Max. :-0.2851 Max. :0.005087 Max. :0.1552
Code
#few details/not so good
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$fidelity_low <- ifelse(dataN14_control$treatment_fidelity == 1, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = fidelity_low)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.2390981 | 0.7235207 | -1.178976 | 1.657173 | 0.7410488 | 0.3948987 |
| 0.51 | 0.2393356 | 0.7222680 | -1.176284 | 1.654955 | 0.7403674 | 0.3946738 |
| 0.52 | 0.2395855 | 0.7210059 | -1.173560 | 1.652731 | 0.7396677 | 0.3944398 |
| 0.53 | 0.2398484 | 0.7197338 | -1.170804 | 1.650501 | 0.7389486 | 0.3941963 |
| 0.54 | 0.2401253 | 0.7184508 | -1.168012 | 1.648263 | 0.7382087 | 0.3939422 |
| 0.55 | 0.2404170 | 0.7171571 | -1.165185 | 1.646019 | 0.7374469 | 0.3936777 |
| 0.56 | 0.2407246 | 0.7158518 | -1.162319 | 1.643768 | 0.7366619 | 0.3934018 |
| 0.57 | 0.2410490 | 0.7145341 | -1.159412 | 1.641510 | 0.7358521 | 0.3931135 |
| 0.58 | 0.2413916 | 0.7132032 | -1.156461 | 1.639244 | 0.7350157 | 0.3928121 |
| 0.59 | 0.2417535 | 0.7118584 | -1.153463 | 1.636970 | 0.7341510 | 0.3924968 |
| 0.60 | 0.2421363 | 0.7104988 | -1.150416 | 1.634688 | 0.7332560 | 0.3921663 |
| 0.61 | 0.2425415 | 0.7091238 | -1.147316 | 1.632399 | 0.7323285 | 0.3918203 |
| 0.62 | 0.2429708 | 0.7077315 | -1.144158 | 1.630099 | 0.7313658 | 0.3914564 |
| 0.63 | 0.2434261 | 0.7063212 | -1.140938 | 1.627790 | 0.7303655 | 0.3910739 |
| 0.64 | 0.2439097 | 0.7048915 | -1.137652 | 1.625472 | 0.7293243 | 0.3906712 |
| 0.65 | 0.2444239 | 0.7034410 | -1.134295 | 1.623143 | 0.7282390 | 0.3902469 |
| 0.66 | 0.2449715 | 0.7019681 | -1.130861 | 1.620804 | 0.7271058 | 0.3897990 |
| 0.67 | 0.2455555 | 0.7004711 | -1.127343 | 1.618454 | 0.7259203 | 0.3893256 |
| 0.68 | 0.2461793 | 0.6989480 | -1.123733 | 1.616092 | 0.7246777 | 0.3888246 |
| 0.69 | 0.2468469 | 0.6973965 | -1.120025 | 1.613719 | 0.7233727 | 0.3882933 |
| 0.70 | 0.2475628 | 0.6958141 | -1.116208 | 1.611333 | 0.7219988 | 0.3877291 |
| 0.71 | 0.2483322 | 0.6941981 | -1.112271 | 1.608936 | 0.7205490 | 0.3871288 |
| 0.72 | 0.2491610 | 0.6925453 | -1.108203 | 1.606525 | 0.7190148 | 0.3864888 |
| 0.73 | 0.2500564 | 0.6908521 | -1.103989 | 1.604102 | 0.7173867 | 0.3858052 |
| 0.74 | 0.2510264 | 0.6891142 | -1.099613 | 1.601665 | 0.7156534 | 0.3850734 |
| 0.75 | 0.2520810 | 0.6873271 | -1.095055 | 1.599217 | 0.7138014 | 0.3842880 |
Code
summary(sensitivity_moderator) #mean = .244, p = .730 ri_t beta se ci.lb
Min. :0.5000 Min. :0.2391 Min. :0.6873 Min. :-1.179
1st Qu.:0.5625 1st Qu.:0.2408 1st Qu.:0.6978 1st Qu.:-1.162
Median :0.6250 Median :0.2432 Median :0.7070 Median :-1.143
Mean :0.6250 Mean :0.2440 Mean :0.7065 Mean :-1.141
3rd Qu.:0.6875 3rd Qu.:0.2467 3rd Qu.:0.7155 3rd Qu.:-1.121
Max. :0.7500 Max. :0.2521 Max. :0.7235 Max. :-1.095
ci.ub pvalue sigma2
Min. :1.599 Min. :0.7138 Min. :0.3843
1st Qu.:1.614 1st Qu.:0.7237 1st Qu.:0.3884
Median :1.629 Median :0.7309 Median :0.3913
Mean :1.629 Mean :0.7297 Mean :0.3907
3rd Qu.:1.643 3rd Qu.:0.7365 3rd Qu.:0.3933
Max. :1.657 Max. :0.7410 Max. :0.3949
Code
#lot of information
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$fidelity_high <- ifelse(dataN14_control$treatment_fidelity == 2, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = fidelity_high)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.7749085 | 0.3696022 | 0.0505016 | 1.499316 | 0.0360289 | 0.2134398 |
| 0.51 | 0.7735492 | 0.3693568 | 0.0496233 | 1.497475 | 0.0362320 | 0.2138227 |
| 0.52 | 0.7721827 | 0.3691073 | 0.0487456 | 1.495620 | 0.0364361 | 0.2142024 |
| 0.53 | 0.7708085 | 0.3688537 | 0.0478685 | 1.493749 | 0.0366411 | 0.2145788 |
| 0.54 | 0.7694262 | 0.3685957 | 0.0469919 | 1.491861 | 0.0368472 | 0.2149515 |
| 0.55 | 0.7680352 | 0.3683330 | 0.0461157 | 1.489955 | 0.0370543 | 0.2153202 |
| 0.56 | 0.7666349 | 0.3680655 | 0.0452398 | 1.488030 | 0.0372625 | 0.2156846 |
| 0.57 | 0.7652246 | 0.3677928 | 0.0443640 | 1.486085 | 0.0374718 | 0.2160444 |
| 0.58 | 0.7638036 | 0.3675146 | 0.0434882 | 1.484119 | 0.0376823 | 0.2163992 |
| 0.59 | 0.7623711 | 0.3672307 | 0.0426120 | 1.482130 | 0.0378939 | 0.2167486 |
| 0.60 | 0.7609261 | 0.3669408 | 0.0417354 | 1.480117 | 0.0381069 | 0.2170922 |
| 0.61 | 0.7594677 | 0.3666444 | 0.0408579 | 1.478077 | 0.0383213 | 0.2174295 |
| 0.62 | 0.7579947 | 0.3663412 | 0.0399792 | 1.476010 | 0.0385372 | 0.2177601 |
| 0.63 | 0.7565058 | 0.3660307 | 0.0390988 | 1.473913 | 0.0387547 | 0.2180833 |
| 0.64 | 0.7549997 | 0.3657125 | 0.0382163 | 1.471783 | 0.0389739 | 0.2183986 |
| 0.65 | 0.7534748 | 0.3653862 | 0.0373311 | 1.469619 | 0.0391951 | 0.2187054 |
| 0.66 | 0.7519293 | 0.3650511 | 0.0364422 | 1.467416 | 0.0394185 | 0.2190030 |
| 0.67 | 0.7503611 | 0.3647067 | 0.0355490 | 1.465173 | 0.0396442 | 0.2192906 |
| 0.68 | 0.7487679 | 0.3643524 | 0.0346503 | 1.462886 | 0.0398727 | 0.2195674 |
| 0.69 | 0.7471472 | 0.3639875 | 0.0337448 | 1.460550 | 0.0401043 | 0.2198326 |
| 0.70 | 0.7454959 | 0.3636112 | 0.0328311 | 1.458161 | 0.0403395 | 0.2200852 |
| 0.71 | 0.7438106 | 0.3632227 | 0.0319072 | 1.455714 | 0.0405788 | 0.2203241 |
| 0.72 | 0.7420874 | 0.3628212 | 0.0309709 | 1.453204 | 0.0408228 | 0.2205483 |
| 0.73 | 0.7403215 | 0.3624057 | 0.0300194 | 1.450624 | 0.0410725 | 0.2207566 |
| 0.74 | 0.7385076 | 0.3619751 | 0.0290494 | 1.447966 | 0.0413288 | 0.2209477 |
| 0.75 | 0.7366395 | 0.3615284 | 0.0280568 | 1.445222 | 0.0415930 | 0.2211201 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.7366 Min. :0.3615 Min. :0.02806
1st Qu.:0.5625 1st Qu.:0.7476 1st Qu.:0.3641 1st Qu.:0.03397
Median :0.6250 Median :0.7573 Median :0.3662 Median :0.03954
Mean :0.6250 Mean :0.7567 Mean :0.3660 Mean :0.03946
3rd Qu.:0.6875 3rd Qu.:0.7663 3rd Qu.:0.3680 3rd Qu.:0.04502
Max. :0.7500 Max. :0.7749 Max. :0.3696 Max. :0.05050
ci.ub pvalue sigma2
Min. :1.445 Min. :0.03603 Min. :0.2134
1st Qu.:1.461 1st Qu.:0.03731 1st Qu.:0.2158
Median :1.475 Median :0.03865 Median :0.2179
Mean :1.474 Mean :0.03870 Mean :0.2177
3rd Qu.:1.488 3rd Qu.:0.04005 3rd Qu.:0.2198
Max. :1.499 Max. :0.04159 Max. :0.2211
Writing tasks
Code
## WRITING TASKS ###########
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric(),
sigma2 = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$test <- as.factor(dataN14_control$writing_tests)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = test)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
sigma2 = RMA$sigma2[1],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue | sigma2 |
|---|---|---|---|---|---|---|
| 0.50 | 0.4653052 | 0.2605106 | -0.0452863 | 0.9758966 | 0.0740786 | 0.2401265 |
| 0.51 | 0.4648472 | 0.2604370 | -0.0455999 | 0.9752943 | 0.0742818 | 0.2402942 |
| 0.52 | 0.4643843 | 0.2603603 | -0.0459125 | 0.9746811 | 0.0744851 | 0.2404571 |
| 0.53 | 0.4639162 | 0.2602803 | -0.0462238 | 0.9740562 | 0.0746887 | 0.2406150 |
| 0.54 | 0.4634426 | 0.2601969 | -0.0465340 | 0.9734191 | 0.0748924 | 0.2407674 |
| 0.55 | 0.4629630 | 0.2601099 | -0.0468430 | 0.9727691 | 0.0750965 | 0.2409141 |
| 0.56 | 0.4624772 | 0.2600191 | -0.0471509 | 0.9721053 | 0.0753008 | 0.2410547 |
| 0.57 | 0.4619847 | 0.2599243 | -0.0474577 | 0.9714270 | 0.0755055 | 0.2411888 |
| 0.58 | 0.4614850 | 0.2598253 | -0.0477633 | 0.9707332 | 0.0757106 | 0.2413160 |
| 0.59 | 0.4609775 | 0.2597218 | -0.0480679 | 0.9700230 | 0.0759162 | 0.2414358 |
| 0.60 | 0.4604618 | 0.2596136 | -0.0483716 | 0.9692952 | 0.0761223 | 0.2415478 |
| 0.61 | 0.4599371 | 0.2595004 | -0.0486744 | 0.9685486 | 0.0763291 | 0.2416514 |
| 0.62 | 0.4594027 | 0.2593819 | -0.0489764 | 0.9677819 | 0.0765367 | 0.2417460 |
| 0.63 | 0.4588579 | 0.2592577 | -0.0492779 | 0.9669937 | 0.0767453 | 0.2418310 |
| 0.64 | 0.4583017 | 0.2591273 | -0.0495785 | 0.9661819 | 0.0769548 | 0.2419054 |
| 0.65 | 0.4577331 | 0.2589907 | -0.0498794 | 0.9653456 | 0.0771659 | 0.2419691 |
| 0.66 | 0.4571510 | 0.2588473 | -0.0501804 | 0.9644825 | 0.0773786 | 0.2420211 |
| 0.67 | 0.4565542 | 0.2586966 | -0.0504819 | 0.9635902 | 0.0775932 | 0.2420604 |
| 0.68 | 0.4559410 | 0.2585381 | -0.0507843 | 0.9626664 | 0.0778102 | 0.2420860 |
| 0.69 | 0.4553101 | 0.2583712 | -0.0510881 | 0.9617082 | 0.0780300 | 0.2420969 |
| 0.70 | 0.4546593 | 0.2581953 | -0.0513941 | 0.9607128 | 0.0782532 | 0.2420920 |
| 0.71 | 0.4539867 | 0.2580097 | -0.0517029 | 0.9596764 | 0.0784803 | 0.2420699 |
| 0.72 | 0.4532898 | 0.2578136 | -0.0520156 | 0.9585953 | 0.0787122 | 0.2420292 |
| 0.73 | 0.4525658 | 0.2576063 | -0.0523332 | 0.9574649 | 0.0789498 | 0.2419683 |
| 0.74 | 0.4518115 | 0.2573867 | -0.0526572 | 0.9562801 | 0.0791943 | 0.2418855 |
| 0.75 | 0.4510230 | 0.2571538 | -0.0529893 | 0.9550352 | 0.0794471 | 0.2417788 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.4510 Min. :0.2572 Min. :-0.05299
1st Qu.:0.5625 1st Qu.:0.4555 1st Qu.:0.2584 1st Qu.:-0.05101
Median :0.6250 Median :0.4591 Median :0.2593 Median :-0.04913
Mean :0.6250 Mean :0.4588 Mean :0.2591 Mean :-0.04912
3rd Qu.:0.6875 3rd Qu.:0.4624 3rd Qu.:0.2600 3rd Qu.:-0.04723
Max. :0.7500 Max. :0.4653 Max. :0.2605 Max. :-0.04529
ci.ub pvalue sigma2
Min. :0.9550 Min. :0.07408 Min. :0.2401
1st Qu.:0.9619 1st Qu.:0.07535 1st Qu.:0.2411
Median :0.9674 Median :0.07664 Median :0.2418
Mean :0.9667 Mean :0.07668 Mean :0.2415
3rd Qu.:0.9719 3rd Qu.:0.07798 3rd Qu.:0.2420
Max. :0.9759 Max. :0.07945 Max. :0.2421
Code
#self-conducted
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$tests_self <- ifelse(dataN14_control$writing_tests == 0, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = tests_self)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue |
|---|---|---|---|---|---|
| 0.50 | -0.7002043 | 0.4530110 | -1.588089 | 0.1876810 | 0.1221849 |
| 0.51 | -0.6990671 | 0.4526691 | -1.586282 | 0.1881482 | 0.1225103 |
| 0.52 | -0.6979268 | 0.4523201 | -1.584458 | 0.1886042 | 0.1228324 |
| 0.53 | -0.6967834 | 0.4519634 | -1.582615 | 0.1890487 | 0.1231512 |
| 0.54 | -0.6956366 | 0.4515989 | -1.580754 | 0.1894809 | 0.1234663 |
| 0.55 | -0.6944863 | 0.4512259 | -1.578873 | 0.1899003 | 0.1237775 |
| 0.56 | -0.6933322 | 0.4508442 | -1.576971 | 0.1903062 | 0.1240845 |
| 0.57 | -0.6921741 | 0.4504532 | -1.575046 | 0.1906980 | 0.1243871 |
| 0.58 | -0.6910117 | 0.4500524 | -1.573098 | 0.1910748 | 0.1246849 |
| 0.59 | -0.6898447 | 0.4496412 | -1.571125 | 0.1914359 | 0.1249777 |
| 0.60 | -0.6886728 | 0.4492191 | -1.569126 | 0.1917805 | 0.1252649 |
| 0.61 | -0.6874955 | 0.4487853 | -1.567098 | 0.1921075 | 0.1255464 |
| 0.62 | -0.6863125 | 0.4483391 | -1.565041 | 0.1924161 | 0.1258216 |
| 0.63 | -0.6851232 | 0.4478798 | -1.562951 | 0.1927050 | 0.1260902 |
| 0.64 | -0.6839271 | 0.4474063 | -1.560827 | 0.1929732 | 0.1263516 |
| 0.65 | -0.6827236 | 0.4469178 | -1.558666 | 0.1932193 | 0.1266053 |
| 0.66 | -0.6815119 | 0.4464133 | -1.556466 | 0.1934420 | 0.1268508 |
| 0.67 | -0.6802914 | 0.4458914 | -1.554222 | 0.1936397 | 0.1270874 |
| 0.68 | -0.6790610 | 0.4453509 | -1.551933 | 0.1938107 | 0.1273146 |
| 0.69 | -0.6778198 | 0.4447903 | -1.549593 | 0.1939532 | 0.1275315 |
| 0.70 | -0.6765667 | 0.4442080 | -1.547198 | 0.1940650 | 0.1277374 |
| 0.71 | -0.6753004 | 0.4436022 | -1.544745 | 0.1941439 | 0.1279312 |
| 0.72 | -0.6740193 | 0.4429707 | -1.542226 | 0.1941873 | 0.1281121 |
| 0.73 | -0.6727219 | 0.4423112 | -1.539636 | 0.1941922 | 0.1282788 |
| 0.74 | -0.6714063 | 0.4416211 | -1.536968 | 0.1941552 | 0.1284301 |
| 0.75 | -0.6700704 | 0.4408973 | -1.534213 | 0.1940724 | 0.1285643 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :-0.7002 Min. :0.4409 Min. :-1.588
1st Qu.:0.5625 1st Qu.:-0.6930 1st Qu.:0.4449 1st Qu.:-1.576
Median :0.6250 Median :-0.6857 Median :0.4481 Median :-1.564
Mean :0.6250 Mean :-0.6855 Mean :0.4477 Mean :-1.563
3rd Qu.:0.6875 3rd Qu.:-0.6781 3rd Qu.:0.4507 3rd Qu.:-1.550
Max. :0.7500 Max. :-0.6701 Max. :0.4530 Max. :-1.534
ci.ub pvalue
Min. :0.1877 Min. :0.1222
1st Qu.:0.1904 1st Qu.:0.1242
Median :0.1926 Median :0.1260
Mean :0.1920 Mean :0.1258
3rd Qu.:0.1939 3rd Qu.:0.1275
Max. :0.1942 Max. :0.1286
Code
#standardized tests
sensitivity_moderator <- data.frame(
ri_t = as.numeric(),
beta = as.numeric(),
se = as.numeric(),
ci.lb = as.numeric(),
ci.ub = as.numeric(),
pvalue = as.numeric()
)
for (ri_t in seq(from = .50, to=.75, by=.01)) {
###compute effect sizes for writing quality
# effect sizes from within-subjects data
datT_es<- escalc(measure = "SMCR",
m1i=datT$m_post, m2i=datT$m_pre,
sd1i=datT$sd_pre, sd2i=datT$sd_post,
ni=datT$ni, ri=rep(ri_t, 14),
slab = datT$slab)
datT_es$id <- datT$id
datC_es <- escalc(measure="SMCR",
m1i=datC$m_post, m2i=datC$m_pre,
sd1i=datC$sd_pre, sd2i=datC$sd_post,
ni=datC$ni, ri=rep((ri_t+.14), 14),
slab = datC$slab)
datC_es$id<-datC$id
dat <- data.frame(yi = datT_es$yi - datC_es$yi, vi = datT_es$vi + datC_es$vi)
dat$id <- c(1,1,1,2,2,3,3,4,4,5,6,6,7,8)
dat$tests_norm <- ifelse(dataN14_control$writing_tests == 2, 1, 0)
# compute the meta-analysis
RMA <- rma.mv(yi, vi, data=dat, random = ~ 1 | id, mods = tests_norm)
# save estimates for sensitivity analysis
sensitivity_moderator <- sensitivity_moderator %>%
add_row(ri_t = ri_t,
beta = RMA$beta[,1][2],
se = RMA$se[2],
ci.lb = RMA$ci.lb[2],
ci.ub = RMA$ci.ub[2],
pvalue = RMA$pval[2]
)
}
sensitivity_moderator %>%
kbl() %>%
kable_styling(bootstrap_options = c("striped", "hover"))| ri_t | beta | se | ci.lb | ci.ub | pvalue |
|---|---|---|---|---|---|
| 0.50 | 0.8935590 | 0.5604653 | -0.2049329 | 1.992051 | 0.1108652 |
| 0.51 | 0.8926921 | 0.5604341 | -0.2057386 | 1.991123 | 0.1111920 |
| 0.52 | 0.8918096 | 0.5603991 | -0.2065525 | 1.990172 | 0.1115234 |
| 0.53 | 0.8909106 | 0.5603602 | -0.2073752 | 1.989196 | 0.1118598 |
| 0.54 | 0.8899938 | 0.5603170 | -0.2082074 | 1.988195 | 0.1122015 |
| 0.55 | 0.8890581 | 0.5602695 | -0.2090500 | 1.987166 | 0.1125489 |
| 0.56 | 0.8881021 | 0.5602174 | -0.2099039 | 1.986108 | 0.1129025 |
| 0.57 | 0.8871243 | 0.5601604 | -0.2107699 | 1.985019 | 0.1132627 |
| 0.58 | 0.8861232 | 0.5600983 | -0.2116493 | 1.983896 | 0.1136301 |
| 0.59 | 0.8850969 | 0.5600307 | -0.2125431 | 1.982737 | 0.1140053 |
| 0.60 | 0.8840434 | 0.5599574 | -0.2134530 | 1.981540 | 0.1143891 |
| 0.61 | 0.8829607 | 0.5598781 | -0.2143803 | 1.980302 | 0.1147820 |
| 0.62 | 0.8818461 | 0.5597924 | -0.2153269 | 1.979019 | 0.1151851 |
| 0.63 | 0.8806971 | 0.5597000 | -0.2162947 | 1.977689 | 0.1155993 |
| 0.64 | 0.8795105 | 0.5596004 | -0.2172861 | 1.976307 | 0.1160257 |
| 0.65 | 0.8782831 | 0.5594932 | -0.2183035 | 1.974870 | 0.1164656 |
| 0.66 | 0.8770108 | 0.5593780 | -0.2193500 | 1.973372 | 0.1169203 |
| 0.67 | 0.8756895 | 0.5592543 | -0.2204288 | 1.971808 | 0.1173916 |
| 0.68 | 0.8743143 | 0.5591217 | -0.2215440 | 1.970173 | 0.1178814 |
| 0.69 | 0.8728796 | 0.5589794 | -0.2227000 | 1.968459 | 0.1183918 |
| 0.70 | 0.8713791 | 0.5588271 | -0.2239019 | 1.966660 | 0.1189254 |
| 0.71 | 0.8698055 | 0.5586640 | -0.2251559 | 1.964767 | 0.1194851 |
| 0.72 | 0.8681505 | 0.5584896 | -0.2264691 | 1.962770 | 0.1200744 |
| 0.73 | 0.8664044 | 0.5583027 | -0.2278488 | 1.960658 | 0.1206971 |
| 0.74 | 0.8645561 | 0.5581036 | -0.2293069 | 1.958419 | 0.1213586 |
| 0.75 | 0.8625924 | 0.5578911 | -0.2308541 | 1.956039 | 0.1220644 |
Code
summary(sensitivity_moderator) ri_t beta se ci.lb
Min. :0.5000 Min. :0.8626 Min. :0.5579 Min. :-0.2309
1st Qu.:0.5625 1st Qu.:0.8732 1st Qu.:0.5590 1st Qu.:-0.2224
Median :0.6250 Median :0.8813 Median :0.5597 Median :-0.2158
Mean :0.6250 Mean :0.8802 Mean :0.5595 Mean :-0.2165
3rd Qu.:0.6875 3rd Qu.:0.8879 3rd Qu.:0.5602 3rd Qu.:-0.2101
Max. :0.7500 Max. :0.8936 Max. :0.5605 Max. :-0.2049
ci.ub pvalue
Min. :1.956 Min. :0.1109
1st Qu.:1.969 1st Qu.:0.1130
Median :1.978 Median :0.1154
Mean :1.977 Mean :0.1158
3rd Qu.:1.986 3rd Qu.:0.1183
Max. :1.992 Max. :0.1221
Computational Environment
R-Version and libraries used.
Code
R version 4.2.2 (2022-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22631)
Matrix products: default
locale:
[1] LC_COLLATE=German_Germany.utf8 LC_CTYPE=German_Germany.utf8
[3] LC_MONETARY=German_Germany.utf8 LC_NUMERIC=C
[5] LC_TIME=German_Germany.utf8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] skimr_2.1.5 xfun_0.36 kableExtra_1.3.4 metafor_3.8-1
[5] metadat_1.2-0 Matrix_1.5-1 haven_2.5.1 here_1.0.1
[9] lubridate_1.9.2 forcats_1.0.0 stringr_1.5.0 dplyr_1.1.0
[13] purrr_1.0.1 readr_2.1.4 tidyr_1.3.0 tibble_3.1.8
[17] ggplot2_3.4.1 tidyverse_2.0.0
loaded via a namespace (and not attached):
[1] tidyselect_1.2.0 repr_1.1.6 lattice_0.22-5 colorspace_2.0-3
[5] vctrs_0.5.2 generics_0.1.3 viridisLite_0.4.1 htmltools_0.5.4
[9] base64enc_0.1-3 yaml_2.3.6 utf8_1.2.2 rlang_1.1.1
[13] pillar_1.8.1 glue_1.6.2 withr_2.5.0 lifecycle_1.0.3
[17] munsell_0.5.0 gtable_0.3.1 rvest_1.0.3 htmlwidgets_1.6.1
[21] evaluate_0.19 knitr_1.41 tzdb_0.3.0 fastmap_1.1.0
[25] fansi_1.0.3 highr_0.10 scales_1.2.1 webshot_0.5.4
[29] jsonlite_1.8.4 mime_0.12 systemfonts_1.0.4 hms_1.1.2
[33] digest_0.6.31 stringi_1.7.8 grid_4.2.2 rprojroot_2.0.3
[37] mathjaxr_1.6-0 cli_3.6.1 tools_4.2.2 magrittr_2.0.3
[41] pkgconfig_2.0.3 ellipsis_0.3.2 xml2_1.3.3 timechange_0.1.1
[45] svglite_2.1.1 rmarkdown_2.19 httr_1.4.4 rstudioapi_0.14
[49] R6_2.5.1 nlme_3.1-160 compiler_4.2.2